Synr: A stable AST for python

Converting from the Python AST

synr.to_ast(program: Union[Any, str], diagnostic_ctx: synr.diagnostic_context.DiagnosticContext, transformer: Optional[synr.transformer.Transformer] = None) Any

Parse an abstract syntax tree from a Python program.

Examples

to_ast() can be used with a given python function or class:

import synr

def my_function(x):
    return x + 2

synr.to_ast(my_function, synr.PrinterDiagnosticContext())

to_ast() can also be used with a string containing python code:

import synr
f = "def my_function(x):\\n    return x + 2"
synr.to_ast(f, synr.PrinterDiagnosticContext())
Parameters
  • program (Union[Any, str]) – A string containing a python program or a python function or class. If a python function or class is used, then line numbers will be localized to the file that the function or class came from.

  • diagnostic_ctx (DiagnosticContext) – A diagnostic context to handle reporting of errors.

  • transformer (Optional[Transformer]) – An optional transformer to apply to the AST after parsing. The transformer allows for converting from synr’s AST to a user specified AST using the same diagnostic context and error handling.

Returns

A synr AST if no transformer was specified and not errors occured. Or an error if one occured. Or the result of applying the transformer to the synr AST.

Return type

Union[synr.ast.Module, Any]

The Synr AST

synr.ast contains definitions of all nodes in the synr AST. This AST is independent of python version: different versions of python will give the same AST. In addition, all nodes contain span information.

class synr.ast.ArrayLiteral(span: synr.ast.Span, values: Sequence[synr.ast.Expr])

Bases: synr.ast.Expr

An array literal in an expression.

Example

[1, 2] in x = [1, 2].

values: Sequence[synr.ast.Expr]
class synr.ast.Assert(span: synr.ast.Span, condition: synr.ast.Expr, msg: Optional[synr.ast.Expr])

Bases: synr.ast.Stmt

An assert statement.

Example

In assert 2 == 2, "Oh no!", condition is 2 == 2, and msg is "Oh no!".

condition: synr.ast.Expr
msg: Optional[synr.ast.Expr]
class synr.ast.Assign(span: synr.ast.Span, lhs: List[synr.ast.Var], ty: Optional[synr.ast.Type], rhs: synr.ast.Expr)

Bases: synr.ast.Stmt

An assignment statement.

Notes

Augmented assignment statements like x += 2 are translated into an operator call and an assignment (i.e. x = x + 2).

Example

In x: int = 2, x is lhs, int is ty, and 2 is rhs.

lhs: List[synr.ast.Var]
rhs: synr.ast.Expr
ty: Optional[synr.ast.Type]
class synr.ast.Attr(span: synr.ast.Span, object: synr.ast.Expr, field: synr.ast.Id)

Bases: synr.ast.Expr

Field access on variable or structure or module.

Examples

In x.y, x is the object and y is the field. For multiple field accesses in a row, they are grouped left to right. For example, x.y.z becomes Attr(Attr(object='x', field='y'), field='z').

field: synr.ast.Id
object: synr.ast.Expr
class synr.ast.Block(span: synr.ast.Span, stmts: List[synr.ast.Stmt])

Bases: synr.ast.Node

A sequence of statements.

Examples

def my_function():
    test_call()
    return 2

Here test_call() and return 2 forms a Block.

if x > 2:
    y = 2

Here y = 2 is a Block.

stmts: List[synr.ast.Stmt]
class synr.ast.BuiltinOp(value)

Bases: enum.Enum

Various built in operators including binary operators, unary operators, and array access.

Examples

+ in 2 + 3. [] in x[2]. or in true or false. > in 3 > 2.

Add = 1
And = 9
BitAnd = 19
BitOr = 18
BitXor = 20
Div = 4
Eq = 11
FloorDiv = 5
GE = 14
GT = 13
Invalid = 24
Invert = 23
LE = 16
LT = 15
Mod = 6
Mul = 3
Not = 17
NotEq = 12
Or = 10
Sub = 2
Subscript = 7
SubscriptAssign = 8
UAdd = 22
USub = 21
class synr.ast.Call(span: synr.ast.Span, func_name: Union[synr.ast.Expr, synr.ast.Op], params: List[synr.ast.Expr], keyword_params: Dict[synr.ast.Expr, synr.ast.Expr])

Bases: synr.ast.Expr

A function call.

Function calls can be:

  • A regular function call of the form x.y(1, 2, z=3). In this case, func_name will contain the function name (x.y), params will contain the arguments (1, 2), and keywords_params will contain any keyword arguments (z=3).

  • A binary operation like x + 2. In this case, func_name will be the binary operation from BuiltinOp, params[0] will be the left hand side (x), and params[1] will be the right hand side 2.

  • A unary operation like not x. In this case, func_name will be the unary operation from BuiltinOp, and params[0] will be the operand (x).

  • An array access like x[2, y]. In this case, func_name will be BuiltinOp.Subscript, params[0] will be the operand (x), params[1] will be a Tuple containing the indices (2, y).

  • An array assignment like x[2, 3] = y. In this case, func_name will be BuiltinOp.SubscriptAssign, params[0] will be the operand (x), params[1] will be a Tuple containing the indices (2, 3), and params[2] will contain the right hand side of the assignment (y).

func_name: Union[synr.ast.Expr, synr.ast.Op]
keyword_params: Dict[synr.ast.Expr, synr.ast.Expr]
params: List[synr.ast.Expr]
class synr.ast.Class(span: synr.ast.Span, name: str, funcs: Dict[str, synr.ast.Function], assignments: List[synr.ast.Assign])

Bases: synr.ast.Node

A class definition.

Example

class MyClass:
    x = 2
    def return_x():
        return x

Here name is MyClass, funcs["return_x"] is def return_x(): return x, and assignments[0] is x = 2.

assignments: List[synr.ast.Assign]
funcs: Dict[str, synr.ast.Function]
name: str
class synr.ast.Constant(span: synr.ast.Span, value: Union[str, None, bool, complex, float, int])

Bases: synr.ast.Expr

A literal value in an expression.

Example

1 in x = 1.

value: Union[str, None, bool, complex, float, int]
class synr.ast.DictLiteral(span: synr.ast.Span, keys: Sequence[synr.ast.Expr], values: Sequence[synr.ast.Expr])

Bases: synr.ast.Expr

A sictionary literal in an expression.

Example

{"x": 2} in x = {"x": 2}.

keys: Sequence[synr.ast.Expr]
values: Sequence[synr.ast.Expr]
class synr.ast.Expr(span: synr.ast.Span)

Bases: synr.ast.Node

Base class of a expression in the AST.

span: synr.ast.Span
class synr.ast.For(span: synr.ast.Span, lhs: List[synr.ast.Var], rhs: synr.ast.Expr, body: synr.ast.Block)

Bases: synr.ast.Stmt

A for statement.

Example

for x in range(2):
    pass

Here lhs will be x, rhs will be range(2), and body will be pass.

body: synr.ast.Block
lhs: List[synr.ast.Var]
rhs: synr.ast.Expr
class synr.ast.Function(span: synr.ast.Span, name: str, params: List[synr.ast.Parameter], ret_type: Optional[synr.ast.Type], body: synr.ast.Block, decorators: List[synr.ast.Expr])

Bases: synr.ast.Stmt

A function declaration.

Example

@F
def my_function(x: int):
    return x + 2

Here name is my_function, x: int is params[0], body is return x + 2, and decorators is [F].

body: synr.ast.Block
decorators: List[synr.ast.Expr]
name: str
params: List[synr.ast.Parameter]
ret_type: Optional[synr.ast.Type]
class synr.ast.Global(span: synr.ast.Span, vars: List[synr.ast.Var])

Bases: synr.ast.Stmt

A global statement.

Example

In global x, y, vars is :code`[x, y]`.

vars: List[synr.ast.Var]
class synr.ast.Id(span: synr.ast.Span, name: str)

Bases: synr.ast.Node

static invalid() synr.ast.Id
name: str
class synr.ast.If(span: synr.ast.Span, condition: synr.ast.Expr, true: synr.ast.Block, false: synr.ast.Block)

Bases: synr.ast.Stmt

An if statement.

Notes

An if statement with elif branches becomes multiple nested if statements.

Examples

if x == 2:
    return x
else:
    return 3

Here condition is x == 2, true is return x, and false is return 3.

Multiple elif statements become nested ifs. For example,

if x == 2:
    return x
elif x == 3:
    return "hi"
else:
    return 3

becomes If('x == 2', 'return x', If('x == 3', 'return "hi"', 'return 3')).

condition: synr.ast.Expr
false: synr.ast.Block
true: synr.ast.Block
class synr.ast.Lambda(span: synr.ast.Span, params: List[synr.ast.Parameter], body: synr.ast.Expr)

Bases: synr.ast.Expr

A lambda expression

Example

In lambda x, y: x + y, x, y are params, x + y is body.

body: synr.ast.Expr
params: List[synr.ast.Parameter]
class synr.ast.Module(span: synr.ast.Span, funcs: Dict[str, Union[synr.ast.Class, synr.ast.Function]])

Bases: synr.ast.Node

A collection of classes and functions.

funcs: Dict[str, Union[synr.ast.Class, synr.ast.Function]]
class synr.ast.Node(span: synr.ast.Span)

Bases: object

Base class of any AST node.

All AST nodes must have a span.

span: synr.ast.Span
class synr.ast.Nonlocal(span: synr.ast.Span, vars: List[synr.ast.Var])

Bases: synr.ast.Stmt

A nonlocal statement.

Example

In nonlocal x, y, vars is :code`[x, y]`.

vars: List[synr.ast.Var]
class synr.ast.Op(span: synr.ast.Span, name: synr.ast.BuiltinOp)

Bases: synr.ast.Node

A builtin operator.

See BuiltinOp for supported operators.

Example

+ in x = 1 + 2.

name: synr.ast.BuiltinOp
class synr.ast.Parameter(span: synr.ast.Span, name: str, ty: Optional[synr.ast.Type])

Bases: synr.ast.Node

Parameter in a function declaration.

Example

x: str in def my_function(x: str).

name: str
ty: Optional[synr.ast.Type]
class synr.ast.Return(span: synr.ast.Span, value: Optional[synr.ast.Expr])

Bases: synr.ast.Stmt

A return statement.

Example

return x.

value: Optional[synr.ast.Expr]
class synr.ast.Slice(span: synr.ast.Span, start: synr.ast.Expr, step: synr.ast.Expr, end: synr.ast.Expr)

Bases: synr.ast.Expr

A slice in an expression.

A slice in a range from [start,end) with step size step.

Notes

If not defined, start is 0, end is -1 and step is 1.

Example

1:2 in x = y[1:2].

end: synr.ast.Expr
start: synr.ast.Expr
step: synr.ast.Expr
class synr.ast.Span(filename: str, start_line: int, start_column: int, end_line: int, end_column: int)

Bases: object

An contiguous interval in a source file from a starting line and column to an ending line and column.

Notes

Line and column numbers are one indexed. The interval spanned is inclusive of the start and exclusive of the end.

between(span: synr.ast.Span) synr.ast.Span

The span starting after the end of the first span and ending before the start of the second span.

Example

>>> Span("", 1, 2, 1, 4).between(Span("", 2, 1, 2, 3))
Span(filename="", start_line=1, start_column=4, end_line=2, end_column=1)
end_column: int
end_line: int
filename: str
static from_ast(filename: str, node: _ast.AST) synr.ast.Span

Extract the span of a python AST node

static invalid() synr.ast.Span

An invalid span

merge(span: synr.ast.Span) synr.ast.Span

Return the span starting from the beginning of the first span and ending at the end of the second span.

Notes

Gaps between the end of the first span and the start of the second span are contained in the merged span. The spans also do not need to be in order.

Example

>>> Span("", 1, 2, 1, 4).merge(Span("", 2, 1, 2, 3))
Span(filename="", start_line=1, start_column=2, end_line=2, end_column=3)
>>> Span("", 2, 3, 2, 1).merge(Span("", 1, 2, 1, 4))
Span(filename="", start_line=1, start_column=2, end_line=2, end_column=3)
start_column: int
start_line: int
subtract(span: synr.ast.Span) synr.ast.Span

The span from the start of the first span to the start of the second span.

Example

>>> Span("", 1, 2, 2, 4).subtract(Span("", 2, 1, 2, 3))
Span(filename="", start_line=1, start_column=2, end_line=2, end_column=1)
static union(spans: Sequence[synr.ast.Span]) synr.ast.Span

A span containing all the given spans with no gaps.

This function is the equivalent to merge with more than two spans.

Example

>>> Span.union(Span("", 1, 2, 1, 4), Span("", 2, 1, 2, 3))
Span(filename="", start_line=1, start_column=2, end_line=2, end_column=3)
class synr.ast.Stmt(span: synr.ast.Span)

Bases: synr.ast.Node

Base class of a statement in the AST.

span: synr.ast.Span
class synr.ast.Tuple(span: synr.ast.Span, values: Sequence[synr.ast.Expr])

Bases: synr.ast.Expr

A tuple in an expression.

Example

(1, 2) in x = (1, 2).

values: Sequence[synr.ast.Expr]
class synr.ast.Type(span: synr.ast.Span)

Bases: synr.ast.Node

Base class of a type in the AST.

span: synr.ast.Span
class synr.ast.TypeApply(span: synr.ast.Span, func_name: synr.ast.Type, params: Sequence[synr.ast.Type])

Bases: synr.ast.Type

Type application.

Example

In x: List[str], List[str] is a TypeCall. In this case, List is the func_name, and str is params[0].

func_name: synr.ast.Type
params: Sequence[synr.ast.Type]
class synr.ast.TypeAttr(span: synr.ast.Span, object: synr.ast.Type, field: synr.ast.Id)

Bases: synr.ast.Type

Field access in a type expression.

This is equivalent to Attr, but for types.

Example

In y: X.Z = 2, object is X and field is Z.

field: synr.ast.Id
object: synr.ast.Type
class synr.ast.TypeCall(span: synr.ast.Span, func_name: Union[synr.ast.Type, synr.ast.BuiltinOp], params: List[synr.ast.Type], keyword_params: Dict[synr.ast.Type, synr.ast.Type])

Bases: synr.ast.Type

A function call in type expression.

This is equivalent to Call, but for type expressions.

Example

In x : List[type(None)], type(None) is a TypeCall. type is the func_name and None is params[0].

func_name: Union[synr.ast.Type, synr.ast.BuiltinOp]
keyword_params: Dict[synr.ast.Type, synr.ast.Type]
params: List[synr.ast.Type]
class synr.ast.TypeConstant(span: synr.ast.Span, value: Union[str, None, bool, complex, float, int])

Bases: synr.ast.Type

A literal value in a type expression.

This is equivalent to Constant, but for type expressions.

Examples

1 in x: Array[1]. None in def my_function() -> None:.

value: Union[str, None, bool, complex, float, int]
class synr.ast.TypeSlice(span: synr.ast.Span, start: synr.ast.Type, step: synr.ast.Type, end: synr.ast.Type)

Bases: synr.ast.Type

A slice in a type expression.

This is equivalent to Slice, but for type expressions.

Example

1:2 in x: Array[1:2].

end: synr.ast.Type
start: synr.ast.Type
step: synr.ast.Type
class synr.ast.TypeTuple(span: synr.ast.Span, values: Sequence[synr.ast.Expr])

Bases: synr.ast.Type

A tuple in a type expression.

Example

(str, int) in x: (str, int).

values: Sequence[synr.ast.Expr]
class synr.ast.TypeVar(span: synr.ast.Span, id: synr.ast.Id)

Bases: synr.ast.Type

Type variable in a type expression.

This is equivalent to Var, but for types.

Example

X in y: X = 2.

id: synr.ast.Id
class synr.ast.UnassignedCall(span: synr.ast.Span, call: synr.ast.Call)

Bases: synr.ast.Stmt

A standalone function call.

Example

def my_function():
    test_call()
    return 2

Here test_call() is an UnassignedCall.

call: synr.ast.Call
class synr.ast.Var(span: synr.ast.Span, id: synr.ast.Id)

Bases: synr.ast.Expr

A variable in an expression.

Examples

x and y in x = y. x in x + 2.

id: synr.ast.Id
static invalid() synr.ast.Var
class synr.ast.While(span: synr.ast.Span, condition: synr.ast.Expr, body: synr.ast.Block)

Bases: synr.ast.Stmt

An while statement.

Examples

while x <= 2:
    pass

Here condition is x <= 2, and body will be pass.

body: synr.ast.Block
condition: synr.ast.Expr
class synr.ast.With(span: synr.ast.Span, lhs: List[synr.ast.Var], rhs: synr.ast.Expr, body: synr.ast.Block)

Bases: synr.ast.Stmt

A with statement.

Example

with open(x) as f:
    pass

Here lhs will be f, rhs will be open(x), and body will be pass.

body: synr.ast.Block
lhs: List[synr.ast.Var]
rhs: synr.ast.Expr

Error Handling

Synr uses a DiagnosticContext to accumulate errors that occurred during parsing. Multiple errors can be accumulated and they are all return after parsing. DiagnosticContext can be subclassed to customize error handling behavior. We also provide a PrinterDiagnosticContext, which prints out errors as they occur.

class synr.DiagnosticContext
add_source(name: str, source: str) None

Add a file with source code to the context. This will be called before any call to emit() that contains a span in this file.

emit(level: str, message: str, span: synr.ast.Span) None

Called when an error has occured.

render() Optional[Any]

Render out all error messages. Can either return a value or raise and execption.

class synr.PrinterDiagnosticContext(sources: Dict[str, Sequence[str]] = NOTHING, errors: List[Tuple[str, str, synr.ast.Span]] = NOTHING)

Bases: synr.diagnostic_context.DiagnosticContext

Simple diagnostic context that prints the error and underlines its location in the source. Raises a RuntimeError on the first error hit.

Transforming the Synr AST to a Different Representation

to_ast() allows you to transform a Synr AST into your desired representation while using the existing Synr error handling infrastructure. First implement a class that inherits from Transformer, then pass an instance of this class to to_ast(). to_ast() will either return your converted class or an errors that occurred.

class synr.Transformer(*args, **kwds)

A visitor to handle user specified transformations on the AST.

do_transform(node: synr.ast.Node, diag: synr.diagnostic_context.DiagnosticContext) Optional[Union[synr.transformer.M, synr.transformer.F, synr.transformer.S, synr.transformer.E, synr.transformer.B, synr.transformer.P, synr.transformer.T]]

Entry point for the transformation.

This is called with the synr AST and the diagnostic context used in parsing the python AST.

error(message, span)

Report an error on a given span.

transform(node: synr.ast.Node) Optional[Union[synr.transformer.M, synr.transformer.F, synr.transformer.S, synr.transformer.E, synr.transformer.B, synr.transformer.P, synr.transformer.T]]

Visitor function.

Call this to recurse into child nodes.