how to use --external option of codeql query run

[Lslightly]

The idea is that I want to add some addition data to codeql database and use these new data as new predicates to enhance the analysis capabilities. I notice the --external option, but I don't know how to use it. So I'm asking here.

Idea: The data can be generated by compiler or other static analysis tools. And they are convertible to CodeQL classes/predicates through user-defined mapping rules. For example, for AST nodes, the location string can be converted to AST node if user-defined adaptor predicates are satisfied. Additional information can be inserted to the added data to avoid conflicts when it is generated by compiler or other static analysis tools.

Take escape analysis in golang as an example. Some variables can be heap allocated decided by compiler. We can dump the definition location, the type and other information about these variables to csv file. When imported by CodeQL, we can define a adaptor using the location string and the type to map csv file to HeapAllocatedVariable class, then we can do more things through CodeQL, like linters or statistical analysis. It's somewhat a problem about fusing the data generated by another extractor.

Issue #9758 also reveals similiar problem I think.

[mbg]

Hi @Lslightly 👋🏻

I notice the --external option, but I don't know how to use it. So I'm asking here.

A relatively minimal example of how to use this feature is the following. In your query, you define the external predicate and use it (named foo here):

external predicate foo(string bar, string baz);

from string a, string b
where foo(a, b)
select a, b

You then create a CSV file with rows for the external predicate, with one column for each parameter. Let's call the following test.csv:

hello, world
goodbye, universe

Now you can run the query with codeql query run path-to-your-query.ql --external=foo=test.csv where foo is the name of the external predicate and test.csv the name of the CSV file with the data. With the above examples, this will produce:

|    a    |     b     |
+---------+-----------+
| hello   |  world    |
| goodbye |  universe |

[Lslightly]

Thanks