Missing Python usage example

[fstarnaud]

Hello !

Coming from Python using libclang bindings, I'm looking for a Quick Start guide to use PASTA.

A simple example such as this, but using PASTA (pypasta) instead of Clang (libclang and clang.cindex) Python bindings would be super useful as a Python bootstrap guide:

import clang.cindex
import sys

idx = clang.cindex.Index.create()

tu = idx.parse('example.cpp')

for diagnostic in tu.diagnostics:
    if diagnostic.severity >= clang.cindex.Diagnostic.Warning:
        print(f"Diagnostic: {diagnostic.spelling}")

print(f"Translation unit spelling: {tu.cursor.spelling}")

for child_cursor in tu.cursor.get_children():
    if child_cursor.kind == clang.cindex.CursorKind.FUNCTION_DECL:
        function_name = child_cursor.spelling
        return_type = child_cursor.result_type.spelling
        location = child_cursor.location
        
        print(f"\nFound Function: {function_name}")
        print(f"  Return type: {return_type}")
        print(f"  Location: line={location.line}, column={location.column}")
        
        for param in child_cursor.get_children():
            if param.kind == clang.cindex.CursorKind.PARM_DECL:
                print(f"  Parameter: {param.spelling} (Type: {param.type.spelling})")

[pgoodman]

Hrmmm.. the wildest example is not a tutorial at all but is this: https://github.com/trailofbits/multiplier/blob/main/bin/Bootstrap/PythonBindings.py

[pgoodman]

Sorry that uses the lift.py script, which is the real example: https://github.com/trailofbits/pasta/blob/master/bin/Inventory/lift.py

[fstarnaud]

Thanks for your answer, Peter.

I had seen this code, but I was missing something fundamental in the design of PASTA (and of the C/C++ toolchain in general, starting with the concept of a "driver" and the LLVM CompilerInvocation abstraction).

Reading through https://blog.trailofbits.com/2023/07/28/the-future-of-clang-based-tooling/:
"PASTA, a C++ and Python wrapper around a large percentage of Clang’s API surface area."
“Clang API isn’t really intended as an external API; it’s an internal API for the rest of Clang."

My understanding was that PASTA is a libclang library replacement. However, looking through the code in this file https://github.com/trailofbits/multiplier/blob/main/bin/Bootstrap/PythonBindings.py, I saw:

// from line 1876 on
cxx: Compiler = Compiler.create_host_compiler(fm, TargetLanguage.CXX)
 argv = ArgumentVector([
     str(cxx.executable_path),
     "-x", "c++",
     "-std=c++20",
     "-isystem", str(args.local_include_dir),
     "-isystem", str(args.system_include_dir),
     "-fsyntax-only",
     "-DGAP_ENABLE_COROUTINES",
     str(args.source_file)])

Which gave me the impression that, to get an AST from a .c / .cpp file, Clang would first need to be invoked from the command-line, by specifying the -fsyntax-only parameter explicitly, and pass its stdout response to PASTA.

This didn't make much sense to be, so thinking about this some more while sifting through the code, I came to question this interpretation, since -fsyntax-only does not output to stdout, -ast-dump is required for that.

So, I think I get it now: to use PASTA, we must first recreate a command-line-like environment, specifying the working directory and arguments by creating a Compiler object.

Thanks,
François

[pgoodman]

Ah so it's more like, it needs something akin to a compile command. With create_host_compiler, it's trying to take features from the compiler that was used to compile pasta itself, which is a bit sketchy if you plan to use pasta in other places. Often you get compile commands from elsewhere. Think of a PASTA Compiler class as just a fancy wrapper around metadata about a compiler. It never actually invokes a compiler by child process; instead it operates similarly to libclang, but focuses on exposing the AST and as many of its methods as possible as directly invocable methods.

[pgoodman]

I think this is another good example, though it doesn't go all the way to ASTs: https://github.com/trailofbits/pasta/tree/master/bin/NormalizeCompileCommands