Fedora 32: ccls program is looking for old version of llvm

I’ve just upgraded from F31 -> F32 and the program ccls is now crashing when I try and use it.

I have found a ticket on the github project that links to a fedora bug report, but they are for F31 and llvm 9 so not sure if it’s the same issue as llvm is on version 10 in F32.

Attempting to debug, I can see that ccls is using /usr/lib64/clang/9.0.1 which no longer exists on my system.

The issue can be reproduced by cloning ccls from github, configuring to do a build, and using the version from the fedora repos on it.

The commands I used and output is below

$ ls
CMakeLists.txt  index_tests  LICENSE  README.md  src  third_party
$ cmake -H. -Brelease -DCMAKE_BUILD_TYPE=Release -DCMAKE_EXPORT_COMPILE_COMMANDS=ON
-- The CXX compiler identification is GNU 10.0.1
-- Check for working CXX compiler: /usr/lib64/ccache/c++
-- Check for working CXX compiler: /usr/lib64/ccache/c++ - works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE  
-- RapidJSON found. Headers: /usr/include
fatal: No names found, cannot describe anything.
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ijc/git_repos/ccls/release
$ ln -s release/compile_commands.json
$ gdb --args /usr/bin/ccls --index=/home/ijc/git_repos/ccls/
GNU gdb (GDB) Fedora 9.1-3.fc32
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/ccls...
Reading symbols from .gnu_debugdata for /usr/bin/ccls...
(No debugging symbols found in .gnu_debugdata for /usr/bin/ccls)
Missing separate debuginfos, use: dnf debuginfo-install ccls-0.20190823.5-3.fc32.x86_64
(gdb) run
Starting program: /usr/bin/ccls --index=/home/ijc/git_repos/ccls/
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib64/libthread_db.so.1".
[New Thread 0x7ffff0760700 (LWP 72261)]
[New Thread 0x7fffeff5f700 (LWP 72262)]
[New Thread 0x7fffef75e700 (LWP 72263)]
12:27:07 ccls           initialize.cc:262 I initialize in directory /home/ijc/git_repos/ccls with uri file:///home/ijc/git_repos/ccls/
12:27:07 ccls           initialize.cc:285 I initializationOptions: {"compilationDatabaseCommand":"","compilationDatabaseDirectory":"","cache":{"directory":".ccls-cache","format":"binary","hierarchicalPath":false,"retainInMemory":2},"capabilities":{"documentOnTypeFormattingProvider":{"firstTriggerCharacter":"}","moreTriggerCharacter":[]},"foldingRangeProvider":true,"workspace":{"workspaceFolders":{"supported":true,"changeNotifications":true}}},"clang":{"excludeArgs":[],"extraArgs":[],"pathMappings":[],"resourceDir":""},"client":{"diagnosticsRelatedInformation":true,"hierarchicalDocumentSymbolSupport":true,"linkSupport":true,"snippetSupport":true},"codeLens":{"localVariables":true},"completion":{"caseSensitivity":2,"detailedLabel":true,"dropOldRequests":true,"duplicateOptional":true,"filterAndSort":true,"include":{"blacklist":[],"maxPathSize":30,"suffixWhitelist":[".h",".hpp",".hh",".inc"],"whitelist":[]},"maxNum":100},"diagnostics":{"blacklist":[],"onChange":1000,"onOpen":0,"onSave":0,"spellChecking":true,"whitelist":[]},"highlight":{"largeFileSize":2097152,"lsRanges":false,"blacklist":[],"whitelist":[]},"index":{"blacklist":[],"comments":2,"initialNoLinkage":false,"initialBlacklist":[],"initialWhitelist":[],"maxInitializerLines":5,"multiVersion":0,"multiVersionBlacklist":[],"multiVersionWhitelist":[],"name":{"suppressUnwrittenScope":false},"onChange":false,"parametersInDeclarations":true,"threads":0,"trackDependency":2,"whitelist":[]},"request":{"timeout":5000},"session":{"maxNum":10},"workspaceSymbol":{"caseSensitivity":1,"maxNum":1000,"sort":true},"xref":{"maxNum":2000}}
12:27:07 ccls           initialize.cc:314 I use -resource-dir=/usr/lib64/clang/9.0.1
12:27:07 ccls           initialize.cc:347 I workspace folder: /home/ijc/git_repos/ccls/
12:27:07 ccls              project.cc:422 I loaded /home/ijc/git_repos/ccls/compile_commands.json
12:27:07 ccls           initialize.cc:372 I start 8 indexers
[New Thread 0x7fffeef5d700 (LWP 72264)]
[New Thread 0x7fffe7fff700 (LWP 72265)]
[New Thread 0x7fffee75c700 (LWP 72266)]
[New Thread 0x7fffedf5b700 (LWP 72267)]
[New Thread 0x7fffed75a700 (LWP 72268)]
[New Thread 0x7fffecf59700 (LWP 72269)]
[New Thread 0x7fffe77fe700 (LWP 72270)]
[New Thread 0x7fffe6ffd700 (LWP 72271)]
12:27:07 ccls           initialize.cc:380 I dispatch initial index requests
entries:    42
pending:    4312:27:07 indexer4         pipeline.cc:340 I parse /home/ijc/git_repos/ccls/src/clang_tu.cc
12:27:07 indexer3         pipeline.cc:340 I parse /home/ijc/git_repos/ccls/src/config.cc
12:27:07 indexer6         pipeline.cc:340 I parse /home/ijc/git_repos/ccls/src/main.cc
12:27:07 indexer0         pipeline.cc:340 I parse /home/ijc/git_repos/ccls/src/include_complete.cc
12:27:07 indexer5         pipeline.cc:340 I parse /home/ijc/git_repos/ccls/src/indexer.cc
12:27:07 indexer7         pipeline.cc:290 I load cache for /home/ijc/git_repos/ccls/third_party/siphash.cc
12:27:07 indexer2         pipeline.cc:340 I parse /home/ijc/git_repos/ccls/src/filesystem.cc
12:27:07 indexer1         pipeline.cc:340 I parse /home/ijc/git_repos/ccls/src/fuzzy_match.cc
12:27:07 indexer7         pipeline.cc:290 I load cache for /home/ijc/git_repos/ccls/src/log.cc
12:27:07 indexer7         pipeline.cc:340 I parse /home/ijc/git_repos/ccls/src/lsp.cc
pending:    41
Thread 11 "indexer6" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffe77fe700 (LWP 72270)]
0x00007ffff6172070 in clang::NestedNameSpecifier::getKind() const () from /usr/lib64/llvm9.0/lib/libclang-cpp.so.9
(gdb) bt
#0  0x00007ffff6172070 in clang::NestedNameSpecifier::getKind() const () from /usr/lib64/llvm9.0/lib/libclang-cpp.so.9
#1  0x00007ffff63e8f4d in clang::Sema::ShouldEnterDeclaratorScope(clang::Scope*, clang::CXXScopeSpec const&) () from /usr/lib64/llvm9.0/lib/libclang-cpp.so.9
#2  0x00007ffff5e9448c in clang::Parser::ParseDirectDeclarator(clang::Declarator&) () from /usr/lib64/llvm9.0/lib/libclang-cpp.so.9
#3  0x00007ffff5e9012b in clang::Parser::ParseDeclGroup(clang::ParsingDeclSpec&, clang::DeclaratorContext, clang::SourceLocation*, clang::Parser::ForRangeInit*) ()
   from /usr/lib64/llvm9.0/lib/libclang-cpp.so.9
#4  0x00007ffff5f1d119 in clang::Parser::ParseDeclarationOrFunctionDefinition(clang::Parser::ParsedAttributesWithRange&, clang::ParsingDeclSpec*, clang::AccessSpecifier)
    () from /usr/lib64/llvm9.0/lib/libclang-cpp.so.9
#5  0x00007ffff5f23d3f in clang::Parser::ParseExternalDeclaration(clang::Parser::ParsedAttributesWithRange&, clang::ParsingDeclSpec*) ()
   from /usr/lib64/llvm9.0/lib/libclang-cpp.so.9
#6  0x00007ffff5ea19ba in clang::Parser::ParseInnerNamespace(llvm::SmallVector<clang::Parser::InnerNamespaceInfo, 4u> const&, unsigned int, clang::SourceLocation&, clang::ParsedAttributes&, clang::BalancedDelimiterTracker&) () from /usr/lib64/llvm9.0/lib/libclang-cpp.so.9
#7  0x00007ffff5ea27f2 in clang::Parser::ParseNamespace(clang::DeclaratorContext, clang::SourceLocation&, clang::SourceLocation) ()
   from /usr/lib64/llvm9.0/lib/libclang-cpp.so.9
#8  0x00007ffff5e917a3 in clang::Parser::ParseDeclaration(clang::DeclaratorContext, clang::SourceLocation&, clang::Parser::ParsedAttributesWithRange&) ()
   from /usr/lib64/llvm9.0/lib/libclang-cpp.so.9
#9  0x00007ffff5f23ca1 in clang::Parser::ParseExternalDeclaration(clang::Parser::ParsedAttributesWithRange&, clang::ParsingDeclSpec*) ()
   from /usr/lib64/llvm9.0/lib/libclang-cpp.so.9
#10 0x00007ffff5ea19ba in clang::Parser::ParseInnerNamespace(llvm::SmallVector<clang::Parser::InnerNamespaceInfo, 4u> const&, unsigned int, clang::SourceLocation&, clang::ParsedAttributes&, clang::BalancedDelimiterTracker&) () from /usr/lib64/llvm9.0/lib/libclang-cpp.so.9
#11 0x00007ffff5ea27f2 in clang::Parser::ParseNamespace(clang::DeclaratorContext, clang::SourceLocation&, clang::SourceLocation) ()
   from /usr/lib64/llvm9.0/lib/libclang-cpp.so.9
#12 0x00007ffff5e917a3 in clang::Parser::ParseDeclaration(clang::DeclaratorContext, clang::SourceLocation&, clang::Parser::ParsedAttributesWithRange&) ()
   from /usr/lib64/llvm9.0/lib/libclang-cpp.so.9
#13 0x00007ffff5f23ca1 in clang::Parser::ParseExternalDeclaration(clang::Parser::ParsedAttributesWithRange&, clang::ParsingDeclSpec*) ()
   from /usr/lib64/llvm9.0/lib/libclang-cpp.so.9
#14 0x00007ffff5f255c5 in clang::Parser::ParseTopLevelDecl(clang::OpaquePtr<clang::DeclGroupRef>&, bool) () from /usr/lib64/llvm9.0/lib/libclang-cpp.so.9
#15 0x00007ffff5e6e829 in clang::ParseAST(clang::Sema&, bool, bool) () from /usr/lib64/llvm9.0/lib/libclang-cpp.so.9
#16 0x00007ffff7266269 in clang::FrontendAction::Execute() () from /usr/lib64/llvm9.0/lib/libclang-cpp.so.9
#17 0x0000555555588721 in void llvm::function_ref<void ()>::callback_fn<ccls::idx::index(ccls::SemaManager*, ccls::WorkingFiles*, ccls::VFS*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<char const*, std::allocator<char const*> > const&, std::vector<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, bool, bool&)::{lambda()#1}>(long) ()
#18 0x00007ffff17a8927 in llvm::CrashRecoveryContext::RunSafely(llvm::function_ref<void ()>) () from /usr/lib64/llvm9.0/lib/libLLVM-9.so
#19 0x0000555555592222 in ccls::idx::index(ccls::SemaManager*, ccls::WorkingFiles*, ccls::VFS*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<char const*, std::allocator<char const*> > const&, std::vector<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, bool, bool&) ()
#20 0x00005555555b4c45 in ccls::pipeline::(anonymous namespace)::indexer_Parse(ccls::SemaManager*, ccls::WorkingFiles*, ccls::Project*, ccls::VFS*, ccls::GroupMatch const&) ()
#21 0x00005555555b7d2c in ccls::pipeline::indexer_Main(ccls::SemaManager*, ccls::VFS*, ccls::Project*, ccls::WorkingFiles*) ()
#22 0x000055555564e22d in ccls::(anonymous namespace)::indexer(void*) ()
#23 0x00007ffff0b1d432 in start_thread () from /usr/lib64/libpthread.so.0
#24 0x00007ffff0a4b9d3 in clone () from /usr/lib64/libc.so.6

I tried installing from source to see if the latest commits fix the issue but I get a link time error related to clang libraries

$ cd release/
$ make -j4
Scanning dependencies of target ccls
[  2%] Building CXX object CMakeFiles/ccls.dir/src/clang_tu.cc.o
[  4%] Building CXX object CMakeFiles/ccls.dir/src/config.cc.o
[  6%] Building CXX object CMakeFiles/ccls.dir/third_party/siphash.cc.o
[  9%] Building CXX object CMakeFiles/ccls.dir/src/filesystem.cc.o
[ 13%] Building CXX object CMakeFiles/ccls.dir/src/fuzzy_match.cc.o
[ 13%] Building CXX object CMakeFiles/ccls.dir/src/main.cc.o
[ 16%] Building CXX object CMakeFiles/ccls.dir/src/include_complete.cc.o
[ 18%] Building CXX object CMakeFiles/ccls.dir/src/indexer.cc.o
[ 20%] Building CXX object CMakeFiles/ccls.dir/src/log.cc.o
[ 25%] Building CXX object CMakeFiles/ccls.dir/src/message_handler.cc.o
[ 25%] Building CXX object CMakeFiles/ccls.dir/src/lsp.cc.o
[ 27%] Building CXX object CMakeFiles/ccls.dir/src/pipeline.cc.o
[ 30%] Building CXX object CMakeFiles/ccls.dir/src/platform_win.cc.o
[ 32%] Building CXX object CMakeFiles/ccls.dir/src/platform_posix.cc.o
[ 34%] Building CXX object CMakeFiles/ccls.dir/src/position.cc.o
[ 37%] Building CXX object CMakeFiles/ccls.dir/src/project.cc.o
[ 39%] Building CXX object CMakeFiles/ccls.dir/src/query.cc.o
In file included from /home/ijc/git_repos/ccls/src/indexer.hh:6,
                 from /home/ijc/git_repos/ccls/src/query.hh:6,
                 from /home/ijc/git_repos/ccls/src/query.cc:4:
/home/ijc/git_repos/ccls/src/lsp.hh: In function ‘std::optional<ccls::SymbolInformation> ccls::getSymbolInfo(ccls::DB*, ccls::SymbolIdx, bool)’:
/home/ijc/git_repos/ccls/src/lsp.hh:179:8: warning: ‘info.ccls::SymbolInformation::kind’ may be used uninitialized in this function [-Wmaybe-uninitialized]
  179 | struct SymbolInformation {
      |        ^~~~~~~~~~~~~~~~~
[ 41%] Building CXX object CMakeFiles/ccls.dir/src/sema_manager.cc.o
[ 44%] Building CXX object CMakeFiles/ccls.dir/src/serializer.cc.o
[ 46%] Building CXX object CMakeFiles/ccls.dir/src/test.cc.o
[ 48%] Building CXX object CMakeFiles/ccls.dir/src/utils.cc.o
[ 51%] Building CXX object CMakeFiles/ccls.dir/src/working_files.cc.o
[ 53%] Building CXX object CMakeFiles/ccls.dir/src/messages/ccls_call.cc.o
[ 55%] Building CXX object CMakeFiles/ccls.dir/src/messages/ccls_info.cc.o
[ 58%] Building CXX object CMakeFiles/ccls.dir/src/messages/ccls_inheritance.cc.o
[ 60%] Building CXX object CMakeFiles/ccls.dir/src/messages/ccls_member.cc.o
[ 62%] Building CXX object CMakeFiles/ccls.dir/src/messages/ccls_navigate.cc.o
[ 65%] Building CXX object CMakeFiles/ccls.dir/src/messages/ccls_reload.cc.o
[ 67%] Building CXX object CMakeFiles/ccls.dir/src/messages/ccls_vars.cc.o
[ 69%] Building CXX object CMakeFiles/ccls.dir/src/messages/initialize.cc.o
[ 72%] Building CXX object CMakeFiles/ccls.dir/src/messages/textDocument_code.cc.o
[ 74%] Building CXX object CMakeFiles/ccls.dir/src/messages/textDocument_completion.cc.o
[ 76%] Building CXX object CMakeFiles/ccls.dir/src/messages/textDocument_definition.cc.o
[ 79%] Building CXX object CMakeFiles/ccls.dir/src/messages/textDocument_did.cc.o
[ 81%] Building CXX object CMakeFiles/ccls.dir/src/messages/textDocument_foldingRange.cc.o
[ 83%] Building CXX object CMakeFiles/ccls.dir/src/messages/textDocument_formatting.cc.o
[ 86%] Building CXX object CMakeFiles/ccls.dir/src/messages/textDocument_document.cc.o
[ 88%] Building CXX object CMakeFiles/ccls.dir/src/messages/textDocument_hover.cc.o
[ 90%] Building CXX object CMakeFiles/ccls.dir/src/messages/textDocument_references.cc.o
[ 93%] Building CXX object CMakeFiles/ccls.dir/src/messages/textDocument_rename.cc.o
[ 95%] Building CXX object CMakeFiles/ccls.dir/src/messages/textDocument_signatureHelp.cc.o
[ 97%] Building CXX object CMakeFiles/ccls.dir/src/messages/workspace.cc.o
[100%] Linking CXX executable ccls
/usr/bin/ld: cannot find -lclangIndex
/usr/bin/ld: cannot find -lclangFormat
/usr/bin/ld: cannot find -lclangTooling
/usr/bin/ld: cannot find -lclangToolingInclusions
/usr/bin/ld: cannot find -lclangToolingCore
/usr/bin/ld: cannot find -lclangFrontend
/usr/bin/ld: cannot find -lclangParse
/usr/bin/ld: cannot find -lclangSerialization
/usr/bin/ld: cannot find -lclangSema
/usr/bin/ld: cannot find -lclangAST
/usr/bin/ld: cannot find -lclangLex
/usr/bin/ld: cannot find -lclangDriver
/usr/bin/ld: cannot find -lclangBasic
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/ccls.dir/build.make:720: ccls] Error 1
make[1]: *** [CMakeFiles/Makefile2:152: CMakeFiles/ccls.dir/all] Error 2
make: *** [Makefile:150: all] Error 2

Anyone got any ideas, or thoughts on how to further debug?

As a new user I could only have 2 links in my original post.

Here is some documentation about how llvm10 on fedora is different to previous version. Not sure if it is helpful or not.

You should file a bug report stating, essentially, the same thing as the F31 bug report, as this is a packaging error.

Thanks! Just wanted to confirm I wasn’t missing anything obvious, bug created.

For anyone else who wants to get ccls running, I ended up installing llvm and clang from source by following these instructions (it takes a long time :smiley:). I could then build/install the ccls repo and use that local version.