Synr: A stable AST for python

Converting from the Python AST

synr.to_ast(program: Union[Any, str], diagnostic_ctx: synr.diagnostic_context.DiagnosticContext, transformer: Optional[synr.transformer.Transformer] = None) → Any

Parse an abstract syntax tree from a Python program.

Examples

to_ast() can be used with a given python function or class:

import synr

def my_function(x):
    return x + 2

synr.to_ast(my_function, synr.PrinterDiagnosticContext())

to_ast() can also be used with a string containing python code:

import synr
f = "def my_function(x):\\n    return x + 2"
synr.to_ast(f, synr.PrinterDiagnosticContext())

Parameters

program (Union[Any, str]) – A string containing a python program or a python function or class. If a python function or class is used, then line numbers will be localized to the file that the function or class came from.
diagnostic_ctx (DiagnosticContext) – A diagnostic context to handle reporting of errors.
transformer (Optional[Transformer]) – An optional transformer to apply to the AST after parsing. The transformer allows for converting from synr’s AST to a user specified AST using the same diagnostic context and error handling.

Returns

A synr AST if no transformer was specified and not errors occured. Or an error if one occured. Or the result of applying the transformer to the synr AST.

Return type

Union[synr.ast.Module, Any]

The Synr AST

synr.ast contains definitions of all nodes in the synr AST. This AST is independent of python version: different versions of python will give the same AST. In addition, all nodes contain span information.

class synr.ast.ArrayLiteral(span: synr.ast.Span, values: Sequence[synr.ast.Expr])

Bases: synr.ast.Expr

An array literal in an expression.

Example

[1, 2] in x = [1, 2].

values: Sequence[synr.ast.Expr]

class synr.ast.Assert(span: synr.ast.Span, condition: synr.ast.Expr, msg: Optional[synr.ast.Expr])

Bases: synr.ast.Stmt

An assert statement.

Example

In assert 2 == 2, "Oh no!", condition is 2 == 2, and msg is "Oh no!".

condition: synr.ast.Expr

msg: Optional[synr.ast.Expr]

class synr.ast.Assign(span: synr.ast.Span, lhs: List[synr.ast.Var], ty: Optional[synr.ast.Type], rhs: synr.ast.Expr)

Bases: synr.ast.Stmt

An assignment statement.

Notes

Augmented assignment statements like x += 2 are translated into an operator call and an assignment (i.e. x = x + 2).

Example

In x: int = 2, x is lhs, int is ty, and 2 is rhs.

lhs: List[synr.ast.Var]

rhs: synr.ast.Expr

ty: Optional[synr.ast.Type]

class synr.ast.Attr(span: synr.ast.Span, object: synr.ast.Expr, field: synr.ast.Id)

Bases: synr.ast.Expr

Field access on variable or structure or module.

Examples

In x.y, x is the object and y is the field. For multiple field accesses in a row, they are grouped left to right. For example, x.y.z becomes Attr(Attr(object='x', field='y'), field='z').

field: synr.ast.Id

object: synr.ast.Expr

class synr.ast.Block(span: synr.ast.Span, stmts: List[synr.ast.Stmt])

Bases: synr.ast.Node

A sequence of statements.

Examples

def my_function():
    test_call()
    return 2

Here test_call() and return 2 forms a Block.

if x > 2:
    y = 2

Here y = 2 is a Block.

stmts: List[synr.ast.Stmt]

class synr.ast.BuiltinOp(value)

Bases: enum.Enum

Various built in operators including binary operators, unary operators, and array access.

Examples

+ in 2 + 3. [] in x[2]. or in true or false. > in 3 > 2.

Add = 1

And = 9

BitAnd = 19

BitOr = 18

BitXor = 20

Div = 4

Eq = 11

FloorDiv = 5

GE = 14

GT = 13

Invalid = 24

Invert = 23

LE = 16

LT = 15

Mod = 6

Mul = 3

Not = 17

NotEq = 12

Or = 10

Sub = 2

Subscript = 7

SubscriptAssign = 8

UAdd = 22

USub = 21

class synr.ast.Call(span: synr.ast.Span, func_name: Union[synr.ast.Expr, synr.ast.Op], params: List[synr.ast.Expr], keyword_params: Dict[synr.ast.Expr, synr.ast.Expr])

Bases: synr.ast.Expr

A function call.

Function calls can be:

A regular function call of the form x.y(1, 2, z=3). In this case, func_name will contain the function name (x.y), params will contain the arguments (1, 2), and keywords_params will contain any keyword arguments (z=3).
A binary operation like x + 2. In this case, func_name will be the binary operation from BuiltinOp, params[0] will be the left hand side (x), and params[1] will be the right hand side 2.
A unary operation like not x. In this case, func_name will be the unary operation from BuiltinOp, and params[0] will be the operand (x).
An array access like x[2, y]. In this case, func_name will be BuiltinOp.Subscript, params[0] will be the operand (x), params[1] will be a Tuple containing the indices (2, y).
An array assignment like x[2, 3] = y. In this case, func_name will be BuiltinOp.SubscriptAssign, params[0] will be the operand (x), params[1] will be a Tuple containing the indices (2, 3), and params[2] will contain the right hand side of the assignment (y).

func_name: Union[synr.ast.Expr, synr.ast.Op]

keyword_params: Dict[synr.ast.Expr, synr.ast.Expr]

params: List[synr.ast.Expr]

class synr.ast.Class(span: synr.ast.Span, name: str, funcs: Dict[str, synr.ast.Function], assignments: List[synr.ast.Assign])

Bases: synr.ast.Node

A class definition.

Example

class MyClass:
    x = 2
    def return_x():
        return x

Here name is MyClass, funcs["return_x"] is def return_x(): return x, and assignments[0] is x = 2.

assignments: List[synr.ast.Assign]

funcs: Dict[str, synr.ast.Function]

name: str

class synr.ast.Constant(span: synr.ast.Span, value: Union[str, None, bool, complex, float, int])

Bases: synr.ast.Expr

A literal value in an expression.

Example

1 in x = 1.

value: Union[str, None, bool, complex, float, int]

class synr.ast.DictLiteral(span: synr.ast.Span, keys: Sequence[synr.ast.Expr], values: Sequence[synr.ast.Expr])

Bases: synr.ast.Expr

A sictionary literal in an expression.

Example

{"x": 2} in x = {"x": 2}.

keys: Sequence[synr.ast.Expr]

values: Sequence[synr.ast.Expr]

class synr.ast.Expr(span: synr.ast.Span)

Bases: synr.ast.Node

Base class of a expression in the AST.

span: synr.ast.Span

class synr.ast.For(span: synr.ast.Span, lhs: List[synr.ast.Var], rhs: synr.ast.Expr, body: synr.ast.Block)

Bases: synr.ast.Stmt

A for statement.

Example

for x in range(2):
    pass

Here lhs will be x, rhs will be range(2), and body will be pass.

body: synr.ast.Block

lhs: List[synr.ast.Var]

rhs: synr.ast.Expr

class synr.ast.Function(span: synr.ast.Span, name: str, params: List[synr.ast.Parameter], ret_type: Optional[synr.ast.Type], body: synr.ast.Block, decorators: List[synr.ast.Expr])

Bases: synr.ast.Stmt

A function declaration.

Example

@F
def my_function(x: int):
    return x + 2

Here name is my_function, x: int is params[0], body is return x + 2, and decorators is [F].

body: synr.ast.Block

decorators: List[synr.ast.Expr]

name: str

params: List[synr.ast.Parameter]

ret_type: Optional[synr.ast.Type]

class synr.ast.Global(span: synr.ast.Span, vars: List[synr.ast.Var])

Bases: synr.ast.Stmt

A global statement.

Example

In global x, y, vars is :code`[x, y]`.

vars: List[synr.ast.Var]

class synr.ast.Id(span: synr.ast.Span, name: str)

Bases: synr.ast.Node

static invalid() → synr.ast.Id

name: str

class synr.ast.If(span: synr.ast.Span, condition: synr.ast.Expr, true: synr.ast.Block, false: synr.ast.Block)

Bases: synr.ast.Stmt

An if statement.

Notes

An if statement with elif branches becomes multiple nested if statements.

Examples

if x == 2:
    return x
else:
    return 3

Here condition is x == 2, true is return x, and false is return 3.

Multiple elif statements become nested ifs. For example,

if x == 2:
    return x
elif x == 3:
    return "hi"
else:
    return 3

becomes If('x == 2', 'return x', If('x == 3', 'return "hi"', 'return 3')).

condition: synr.ast.Expr

false: synr.ast.Block

true: synr.ast.Block

class synr.ast.Lambda(span: synr.ast.Span, params: List[synr.ast.Parameter], body: synr.ast.Expr)

Bases: synr.ast.Expr

A lambda expression

Example

In lambda x, y: x + y, x, y are params, x + y is body.

body: synr.ast.Expr

params: List[synr.ast.Parameter]

class synr.ast.Module(span: synr.ast.Span, funcs: Dict[str, Union[synr.ast.Class, synr.ast.Function]])

Bases: synr.ast.Node

A collection of classes and functions.

funcs: Dict[str, Union[synr.ast.Class, synr.ast.Function]]

class synr.ast.Node(span: synr.ast.Span)

Bases: object

Base class of any AST node.

All AST nodes must have a span.

span: synr.ast.Span

class synr.ast.Nonlocal(span: synr.ast.Span, vars: List[synr.ast.Var])

Bases: synr.ast.Stmt

A nonlocal statement.

Example

In nonlocal x, y, vars is :code`[x, y]`.

vars: List[synr.ast.Var]

class synr.ast.Op(span: synr.ast.Span, name: synr.ast.BuiltinOp)

Bases: synr.ast.Node

A builtin operator.

See BuiltinOp for supported operators.

Example

+ in x = 1 + 2.

name: synr.ast.BuiltinOp

class synr.ast.Parameter(span: synr.ast.Span, name: str, ty: Optional[synr.ast.Type])

Bases: synr.ast.Node

Parameter in a function declaration.

Example

x: str in def my_function(x: str).

name: str

ty: Optional[synr.ast.Type]

class synr.ast.Return(span: synr.ast.Span, value: Optional[synr.ast.Expr])

Bases: synr.ast.Stmt

A return statement.

Example

return x.

value: Optional[synr.ast.Expr]

class synr.ast.Slice(span: synr.ast.Span, start: synr.ast.Expr, step: synr.ast.Expr, end: synr.ast.Expr)

Bases: synr.ast.Expr

A slice in an expression.

A slice in a range from [start,end) with step size step.

Notes

If not defined, start is 0, end is -1 and step is 1.

Example

1:2 in x = y[1:2].

end: synr.ast.Expr

start: synr.ast.Expr

step: synr.ast.Expr

class synr.ast.Span(filename: str, start_line: int, start_column: int, end_line: int, end_column: int)

Bases: object

An contiguous interval in a source file from a starting line and column to an ending line and column.

Notes

Line and column numbers are one indexed. The interval spanned is inclusive of the start and exclusive of the end.

between(span: synr.ast.Span) → synr.ast.Span

The span starting after the end of the first span and ending before the start of the second span.

Example

>>> Span("", 1, 2, 1, 4).between(Span("", 2, 1, 2, 3))
Span(filename="", start_line=1, start_column=4, end_line=2, end_column=1)

end_column: int

end_line: int

filename: str

static from_ast(filename: str, node: _ast.AST) → synr.ast.Span: Extract the span of a python AST node

static invalid() → synr.ast.Span: An invalid span

merge(span: synr.ast.Span) → synr.ast.Span

Return the span starting from the beginning of the first span and ending at the end of the second span.

Notes

Gaps between the end of the first span and the start of the second span are contained in the merged span. The spans also do not need to be in order.

Example

>>> Span("", 1, 2, 1, 4).merge(Span("", 2, 1, 2, 3))
Span(filename="", start_line=1, start_column=2, end_line=2, end_column=3)
>>> Span("", 2, 3, 2, 1).merge(Span("", 1, 2, 1, 4))
Span(filename="", start_line=1, start_column=2, end_line=2, end_column=3)

start_column: int

start_line: int

subtract(span: synr.ast.Span) → synr.ast.Span

The span from the start of the first span to the start of the second span.

Example

>>> Span("", 1, 2, 2, 4).subtract(Span("", 2, 1, 2, 3))
Span(filename="", start_line=1, start_column=2, end_line=2, end_column=1)

static union(spans: Sequence[synr.ast.Span]) → synr.ast.Span

A span containing all the given spans with no gaps.

This function is the equivalent to merge with more than two spans.

Example

>>> Span.union(Span("", 1, 2, 1, 4), Span("", 2, 1, 2, 3))
Span(filename="", start_line=1, start_column=2, end_line=2, end_column=3)

class synr.ast.Stmt(span: synr.ast.Span)

Bases: synr.ast.Node

Base class of a statement in the AST.

span: synr.ast.Span

class synr.ast.Tuple(span: synr.ast.Span, values: Sequence[synr.ast.Expr])

Bases: synr.ast.Expr

A tuple in an expression.

Example

(1, 2) in x = (1, 2).

values: Sequence[synr.ast.Expr]

class synr.ast.Type(span: synr.ast.Span)

Bases: synr.ast.Node

Base class of a type in the AST.

span: synr.ast.Span

class synr.ast.TypeApply(span: synr.ast.Span, func_name: synr.ast.Type, params: Sequence[synr.ast.Type])

Bases: synr.ast.Type

Type application.

Example

In x: List[str], List[str] is a TypeCall. In this case, List is the func_name, and str is params[0].

func_name: synr.ast.Type

params: Sequence[synr.ast.Type]

class synr.ast.TypeAttr(span: synr.ast.Span, object: synr.ast.Type, field: synr.ast.Id)

Bases: synr.ast.Type

Field access in a type expression.

This is equivalent to Attr, but for types.

Example

In y: X.Z = 2, object is X and field is Z.

field: synr.ast.Id

object: synr.ast.Type

class synr.ast.TypeCall(span: synr.ast.Span, func_name: Union[synr.ast.Type, synr.ast.BuiltinOp], params: List[synr.ast.Type], keyword_params: Dict[synr.ast.Type, synr.ast.Type])

Bases: synr.ast.Type

A function call in type expression.

This is equivalent to Call, but for type expressions.

Example

In x : List[type(None)], type(None) is a TypeCall. type is the func_name and None is params[0].

func_name: Union[synr.ast.Type, synr.ast.BuiltinOp]

keyword_params: Dict[synr.ast.Type, synr.ast.Type]

params: List[synr.ast.Type]

class synr.ast.TypeConstant(span: synr.ast.Span, value: Union[str, None, bool, complex, float, int])

Bases: synr.ast.Type

A literal value in a type expression.

This is equivalent to Constant, but for type expressions.

Examples

1 in x: Array[1]. None in def my_function() -> None:.

value: Union[str, None, bool, complex, float, int]

class synr.ast.TypeSlice(span: synr.ast.Span, start: synr.ast.Type, step: synr.ast.Type, end: synr.ast.Type)

Bases: synr.ast.Type

A slice in a type expression.

This is equivalent to Slice, but for type expressions.

Example

1:2 in x: Array[1:2].

end: synr.ast.Type

start: synr.ast.Type

step: synr.ast.Type

class synr.ast.TypeTuple(span: synr.ast.Span, values: Sequence[synr.ast.Expr])

Bases: synr.ast.Type

A tuple in a type expression.

Example

(str, int) in x: (str, int).

values: Sequence[synr.ast.Expr]

class synr.ast.TypeVar(span: synr.ast.Span, id: synr.ast.Id)

Bases: synr.ast.Type

Type variable in a type expression.

This is equivalent to Var, but for types.

Example

X in y: X = 2.

id: synr.ast.Id

class synr.ast.UnassignedCall(span: synr.ast.Span, call: synr.ast.Call)

Bases: synr.ast.Stmt

A standalone function call.

Example

def my_function():
    test_call()
    return 2

Here test_call() is an UnassignedCall.

call: synr.ast.Call

class synr.ast.Var(span: synr.ast.Span, id: synr.ast.Id)

Bases: synr.ast.Expr

A variable in an expression.

Examples

x and y in x = y. x in x + 2.

id: synr.ast.Id

static invalid() → synr.ast.Var

class synr.ast.While(span: synr.ast.Span, condition: synr.ast.Expr, body: synr.ast.Block)

Bases: synr.ast.Stmt

An while statement.

Examples

while x <= 2:
    pass

Here condition is x <= 2, and body will be pass.

body: synr.ast.Block

condition: synr.ast.Expr

class synr.ast.With(span: synr.ast.Span, lhs: List[synr.ast.Var], rhs: synr.ast.Expr, body: synr.ast.Block)

Bases: synr.ast.Stmt

A with statement.

Example

with open(x) as f:
    pass

Here lhs will be f, rhs will be open(x), and body will be pass.

body: synr.ast.Block

lhs: List[synr.ast.Var]

rhs: synr.ast.Expr

Error Handling

Synr uses a DiagnosticContext to accumulate errors that occurred during parsing. Multiple errors can be accumulated and they are all return after parsing. DiagnosticContext can be subclassed to customize error handling behavior. We also provide a PrinterDiagnosticContext, which prints out errors as they occur.

class synr.DiagnosticContext

add_source(name: str, source: str) → None: Add a file with source code to the context. This will be called before any call to emit() that contains a span in this file.

emit(level: str, message: str, span: synr.ast.Span) → None: Called when an error has occured.

render() → Optional[Any]: Render out all error messages. Can either return a value or raise and execption.

class synr.PrinterDiagnosticContext(sources: Dict[str, Sequence[str]] = NOTHING, errors: List[Tuple[str, str, synr.ast.Span]] = NOTHING)

Bases: synr.diagnostic_context.DiagnosticContext

Simple diagnostic context that prints the error and underlines its location in the source. Raises a RuntimeError on the first error hit.

Transforming the Synr AST to a Different Representation

to_ast() allows you to transform a Synr AST into your desired representation while using the existing Synr error handling infrastructure. First implement a class that inherits from Transformer, then pass an instance of this class to to_ast(). to_ast() will either return your converted class or an errors that occurred.

class synr.Transformer(*args, **kwds)

A visitor to handle user specified transformations on the AST.

do_transform(node: synr.ast.Node, diag: synr.diagnostic_context.DiagnosticContext) → Optional[Union[synr.transformer.M, synr.transformer.F, synr.transformer.S, synr.transformer.E, synr.transformer.B, synr.transformer.P, synr.transformer.T]]

Entry point for the transformation.

This is called with the synr AST and the diagnostic context used in parsing the python AST.

error(message, span): Report an error on a given span.

transform(node: synr.ast.Node) → Optional[Union[synr.transformer.M, synr.transformer.F, synr.transformer.S, synr.transformer.E, synr.transformer.B, synr.transformer.P, synr.transformer.T]]

Visitor function.

Call this to recurse into child nodes.