Synr: A stable AST for python
Converting from the Python AST
- synr.to_ast(program: Union[Any, str], diagnostic_ctx: synr.diagnostic_context.DiagnosticContext, transformer: Optional[synr.transformer.Transformer] = None) Any
Parse an abstract syntax tree from a Python program.
Examples
to_ast()
can be used with a given python function or class:import synr def my_function(x): return x + 2 synr.to_ast(my_function, synr.PrinterDiagnosticContext())
to_ast()
can also be used with a string containing python code:import synr f = "def my_function(x):\\n return x + 2" synr.to_ast(f, synr.PrinterDiagnosticContext())
- Parameters
program (Union[Any, str]) – A string containing a python program or a python function or class. If a python function or class is used, then line numbers will be localized to the file that the function or class came from.
diagnostic_ctx (DiagnosticContext) – A diagnostic context to handle reporting of errors.
transformer (Optional[Transformer]) – An optional transformer to apply to the AST after parsing. The transformer allows for converting from synr’s AST to a user specified AST using the same diagnostic context and error handling.
- Returns
A synr AST if no transformer was specified and not errors occured. Or an error if one occured. Or the result of applying the transformer to the synr AST.
- Return type
Union[synr.ast.Module, Any]
The Synr AST
synr.ast contains definitions of all nodes in the synr AST. This AST is independent of python version: different versions of python will give the same AST. In addition, all nodes contain span information.
- class synr.ast.ArrayLiteral(span: synr.ast.Span, values: Sequence[synr.ast.Expr])
Bases:
synr.ast.Expr
An array literal in an expression.
Example
[1, 2]
inx = [1, 2]
.- values: Sequence[synr.ast.Expr]
- class synr.ast.Assert(span: synr.ast.Span, condition: synr.ast.Expr, msg: Optional[synr.ast.Expr])
Bases:
synr.ast.Stmt
An assert statement.
Example
In
assert 2 == 2, "Oh no!"
,condition
is2 == 2
, andmsg
is"Oh no!"
.- condition: synr.ast.Expr
- msg: Optional[synr.ast.Expr]
- class synr.ast.Assign(span: synr.ast.Span, lhs: List[synr.ast.Var], ty: Optional[synr.ast.Type], rhs: synr.ast.Expr)
Bases:
synr.ast.Stmt
An assignment statement.
Notes
Augmented assignment statements like
x += 2
are translated into an operator call and an assignment (i.e. x = x + 2).Example
In
x: int = 2
,x
islhs
,int
isty
, and2
isrhs
.- lhs: List[synr.ast.Var]
- rhs: synr.ast.Expr
- ty: Optional[synr.ast.Type]
- class synr.ast.Attr(span: synr.ast.Span, object: synr.ast.Expr, field: synr.ast.Id)
Bases:
synr.ast.Expr
Field access on variable or structure or module.
Examples
In
x.y
,x
is theobject
andy
is thefield
. For multiple field accesses in a row, they are grouped left to right. For example,x.y.z
becomesAttr(Attr(object='x', field='y'), field='z')
.- field: synr.ast.Id
- object: synr.ast.Expr
- class synr.ast.Block(span: synr.ast.Span, stmts: List[synr.ast.Stmt])
Bases:
synr.ast.Node
A sequence of statements.
Examples
def my_function(): test_call() return 2
Here
test_call()
andreturn 2
forms aBlock
.if x > 2: y = 2
Here
y = 2
is aBlock
.- stmts: List[synr.ast.Stmt]
- class synr.ast.BuiltinOp(value)
Bases:
enum.Enum
Various built in operators including binary operators, unary operators, and array access.
Examples
+
in2 + 3
.[]
inx[2]
.or
intrue or false
.>
in3 > 2
.- Add = 1
- And = 9
- BitAnd = 19
- BitOr = 18
- BitXor = 20
- Div = 4
- Eq = 11
- FloorDiv = 5
- GE = 14
- GT = 13
- Invalid = 24
- Invert = 23
- LE = 16
- LT = 15
- Mod = 6
- Mul = 3
- Not = 17
- NotEq = 12
- Or = 10
- Sub = 2
- Subscript = 7
- SubscriptAssign = 8
- UAdd = 22
- USub = 21
- class synr.ast.Call(span: synr.ast.Span, func_name: Union[synr.ast.Expr, synr.ast.Op], params: List[synr.ast.Expr], keyword_params: Dict[synr.ast.Expr, synr.ast.Expr])
Bases:
synr.ast.Expr
A function call.
Function calls can be:
A regular function call of the form
x.y(1, 2, z=3)
. In this case,func_name
will contain the function name (x.y),params
will contain the arguments (1, 2), andkeywords_params
will contain any keyword arguments (z=3).A binary operation like
x + 2
. In this case,func_name
will be the binary operation fromBuiltinOp
,params[0]
will be the left hand side (x), andparams[1]
will be the right hand side2
.A unary operation like
not x
. In this case,func_name
will be the unary operation fromBuiltinOp
, andparams[0]
will be the operand (x).An array access like
x[2, y]
. In this case,func_name
will beBuiltinOp.Subscript
,params[0]
will be the operand (x),params[1]
will be aTuple
containing the indices (2, y).An array assignment like
x[2, 3] = y
. In this case,func_name
will beBuiltinOp.SubscriptAssign
,params[0]
will be the operand (x),params[1]
will be aTuple
containing the indices (2, 3), andparams[2]
will contain the right hand side of the assignment (y).
- func_name: Union[synr.ast.Expr, synr.ast.Op]
- keyword_params: Dict[synr.ast.Expr, synr.ast.Expr]
- params: List[synr.ast.Expr]
- class synr.ast.Class(span: synr.ast.Span, name: str, funcs: Dict[str, synr.ast.Function], assignments: List[synr.ast.Assign])
Bases:
synr.ast.Node
A class definition.
Example
class MyClass: x = 2 def return_x(): return x
Here
name
isMyClass
,funcs["return_x"]
isdef return_x(): return x
, andassignments[0]
isx = 2
.- assignments: List[synr.ast.Assign]
- funcs: Dict[str, synr.ast.Function]
- name: str
- class synr.ast.Constant(span: synr.ast.Span, value: Union[str, None, bool, complex, float, int])
Bases:
synr.ast.Expr
A literal value in an expression.
Example
1
inx = 1
.- value: Union[str, None, bool, complex, float, int]
- class synr.ast.DictLiteral(span: synr.ast.Span, keys: Sequence[synr.ast.Expr], values: Sequence[synr.ast.Expr])
Bases:
synr.ast.Expr
A sictionary literal in an expression.
Example
{"x": 2}
inx = {"x": 2}
.- keys: Sequence[synr.ast.Expr]
- values: Sequence[synr.ast.Expr]
- class synr.ast.Expr(span: synr.ast.Span)
Bases:
synr.ast.Node
Base class of a expression in the AST.
- span: synr.ast.Span
- class synr.ast.For(span: synr.ast.Span, lhs: List[synr.ast.Var], rhs: synr.ast.Expr, body: synr.ast.Block)
Bases:
synr.ast.Stmt
A for statement.
Example
for x in range(2): pass
Here
lhs
will bex
,rhs
will berange(2)
, andbody
will bepass
.- body: synr.ast.Block
- lhs: List[synr.ast.Var]
- rhs: synr.ast.Expr
- class synr.ast.Function(span: synr.ast.Span, name: str, params: List[synr.ast.Parameter], ret_type: Optional[synr.ast.Type], body: synr.ast.Block, decorators: List[synr.ast.Expr])
Bases:
synr.ast.Stmt
A function declaration.
Example
@F def my_function(x: int): return x + 2
Here
name
ismy_function
,x: int
isparams[0]
,body
isreturn x + 2
, anddecorators
is[F]
.- body: synr.ast.Block
- decorators: List[synr.ast.Expr]
- name: str
- params: List[synr.ast.Parameter]
- ret_type: Optional[synr.ast.Type]
- class synr.ast.Global(span: synr.ast.Span, vars: List[synr.ast.Var])
Bases:
synr.ast.Stmt
A global statement.
Example
In
global x, y
,vars
is :code`[x, y]`.- vars: List[synr.ast.Var]
- class synr.ast.Id(span: synr.ast.Span, name: str)
Bases:
synr.ast.Node
- static invalid() synr.ast.Id
- name: str
- class synr.ast.If(span: synr.ast.Span, condition: synr.ast.Expr, true: synr.ast.Block, false: synr.ast.Block)
Bases:
synr.ast.Stmt
An if statement.
Notes
An if statement with
elif
branches becomes multiple nested if statements.Examples
if x == 2: return x else: return 3
Here
condition
isx == 2
,true
isreturn x
, andfalse
isreturn 3
.Multiple
elif
statements become nested ifs. For example,if x == 2: return x elif x == 3: return "hi" else: return 3
becomes
If('x == 2', 'return x', If('x == 3', 'return "hi"', 'return 3'))
.- condition: synr.ast.Expr
- false: synr.ast.Block
- true: synr.ast.Block
- class synr.ast.Lambda(span: synr.ast.Span, params: List[synr.ast.Parameter], body: synr.ast.Expr)
Bases:
synr.ast.Expr
A lambda expression
Example
In
lambda x, y: x + y
,x, y
areparams
,x + y
isbody
.- body: synr.ast.Expr
- params: List[synr.ast.Parameter]
- class synr.ast.Module(span: synr.ast.Span, funcs: Dict[str, Union[synr.ast.Class, synr.ast.Function]])
Bases:
synr.ast.Node
A collection of classes and functions.
- funcs: Dict[str, Union[synr.ast.Class, synr.ast.Function]]
- class synr.ast.Node(span: synr.ast.Span)
Bases:
object
Base class of any AST node.
All AST nodes must have a span.
- span: synr.ast.Span
- class synr.ast.Nonlocal(span: synr.ast.Span, vars: List[synr.ast.Var])
Bases:
synr.ast.Stmt
A nonlocal statement.
Example
In
nonlocal x, y
,vars
is :code`[x, y]`.- vars: List[synr.ast.Var]
- class synr.ast.Op(span: synr.ast.Span, name: synr.ast.BuiltinOp)
Bases:
synr.ast.Node
A builtin operator.
See
BuiltinOp
for supported operators.Example
+
inx = 1 + 2
.- name: synr.ast.BuiltinOp
- class synr.ast.Parameter(span: synr.ast.Span, name: str, ty: Optional[synr.ast.Type])
Bases:
synr.ast.Node
Parameter in a function declaration.
Example
x: str
indef my_function(x: str)
.- name: str
- ty: Optional[synr.ast.Type]
- class synr.ast.Return(span: synr.ast.Span, value: Optional[synr.ast.Expr])
Bases:
synr.ast.Stmt
A return statement.
Example
return x
.- value: Optional[synr.ast.Expr]
- class synr.ast.Slice(span: synr.ast.Span, start: synr.ast.Expr, step: synr.ast.Expr, end: synr.ast.Expr)
Bases:
synr.ast.Expr
A slice in an expression.
A slice in a range from [start,end) with step size step.
Notes
If not defined, start is 0, end is -1 and step is 1.
Example
1:2
inx = y[1:2]
.- end: synr.ast.Expr
- start: synr.ast.Expr
- step: synr.ast.Expr
- class synr.ast.Span(filename: str, start_line: int, start_column: int, end_line: int, end_column: int)
Bases:
object
An contiguous interval in a source file from a starting line and column to an ending line and column.
Notes
Line and column numbers are one indexed. The interval spanned is inclusive of the start and exclusive of the end.
- between(span: synr.ast.Span) synr.ast.Span
The span starting after the end of the first span and ending before the start of the second span.
Example
>>> Span("", 1, 2, 1, 4).between(Span("", 2, 1, 2, 3)) Span(filename="", start_line=1, start_column=4, end_line=2, end_column=1)
- end_column: int
- end_line: int
- filename: str
- static from_ast(filename: str, node: _ast.AST) synr.ast.Span
Extract the span of a python AST node
- static invalid() synr.ast.Span
An invalid span
- merge(span: synr.ast.Span) synr.ast.Span
Return the span starting from the beginning of the first span and ending at the end of the second span.
Notes
Gaps between the end of the first span and the start of the second span are contained in the merged span. The spans also do not need to be in order.
Example
>>> Span("", 1, 2, 1, 4).merge(Span("", 2, 1, 2, 3)) Span(filename="", start_line=1, start_column=2, end_line=2, end_column=3) >>> Span("", 2, 3, 2, 1).merge(Span("", 1, 2, 1, 4)) Span(filename="", start_line=1, start_column=2, end_line=2, end_column=3)
- start_column: int
- start_line: int
- subtract(span: synr.ast.Span) synr.ast.Span
The span from the start of the first span to the start of the second span.
Example
>>> Span("", 1, 2, 2, 4).subtract(Span("", 2, 1, 2, 3)) Span(filename="", start_line=1, start_column=2, end_line=2, end_column=1)
- static union(spans: Sequence[synr.ast.Span]) synr.ast.Span
A span containing all the given spans with no gaps.
This function is the equivalent to merge with more than two spans.
Example
>>> Span.union(Span("", 1, 2, 1, 4), Span("", 2, 1, 2, 3)) Span(filename="", start_line=1, start_column=2, end_line=2, end_column=3)
- class synr.ast.Stmt(span: synr.ast.Span)
Bases:
synr.ast.Node
Base class of a statement in the AST.
- span: synr.ast.Span
- class synr.ast.Tuple(span: synr.ast.Span, values: Sequence[synr.ast.Expr])
Bases:
synr.ast.Expr
A tuple in an expression.
Example
(1, 2)
inx = (1, 2)
.- values: Sequence[synr.ast.Expr]
- class synr.ast.Type(span: synr.ast.Span)
Bases:
synr.ast.Node
Base class of a type in the AST.
- span: synr.ast.Span
- class synr.ast.TypeApply(span: synr.ast.Span, func_name: synr.ast.Type, params: Sequence[synr.ast.Type])
Bases:
synr.ast.Type
Type application.
Example
In
x: List[str]
,List[str]
is aTypeCall
. In this case,List
is thefunc_name
, andstr
isparams[0]
.- func_name: synr.ast.Type
- params: Sequence[synr.ast.Type]
- class synr.ast.TypeAttr(span: synr.ast.Span, object: synr.ast.Type, field: synr.ast.Id)
Bases:
synr.ast.Type
Field access in a type expression.
This is equivalent to
Attr
, but for types.Example
In
y: X.Z = 2
,object
isX
andfield
isZ
.- field: synr.ast.Id
- object: synr.ast.Type
- class synr.ast.TypeCall(span: synr.ast.Span, func_name: Union[synr.ast.Type, synr.ast.BuiltinOp], params: List[synr.ast.Type], keyword_params: Dict[synr.ast.Type, synr.ast.Type])
Bases:
synr.ast.Type
A function call in type expression.
This is equivalent to
Call
, but for type expressions.Example
In
x : List[type(None)]
,type(None)
is aTypeCall
.type
is thefunc_name
andNone
isparams[0]
.- func_name: Union[synr.ast.Type, synr.ast.BuiltinOp]
- keyword_params: Dict[synr.ast.Type, synr.ast.Type]
- params: List[synr.ast.Type]
- class synr.ast.TypeConstant(span: synr.ast.Span, value: Union[str, None, bool, complex, float, int])
Bases:
synr.ast.Type
A literal value in a type expression.
This is equivalent to
Constant
, but for type expressions.Examples
1
inx: Array[1]
.None
indef my_function() -> None:
.- value: Union[str, None, bool, complex, float, int]
- class synr.ast.TypeSlice(span: synr.ast.Span, start: synr.ast.Type, step: synr.ast.Type, end: synr.ast.Type)
Bases:
synr.ast.Type
A slice in a type expression.
This is equivalent to
Slice
, but for type expressions.Example
1:2
inx: Array[1:2]
.- end: synr.ast.Type
- start: synr.ast.Type
- step: synr.ast.Type
- class synr.ast.TypeTuple(span: synr.ast.Span, values: Sequence[synr.ast.Expr])
Bases:
synr.ast.Type
A tuple in a type expression.
Example
(str, int)
inx: (str, int)
.- values: Sequence[synr.ast.Expr]
- class synr.ast.TypeVar(span: synr.ast.Span, id: synr.ast.Id)
Bases:
synr.ast.Type
Type variable in a type expression.
This is equivalent to
Var
, but for types.Example
X
iny: X = 2
.- id: synr.ast.Id
- class synr.ast.UnassignedCall(span: synr.ast.Span, call: synr.ast.Call)
Bases:
synr.ast.Stmt
A standalone function call.
Example
def my_function(): test_call() return 2
Here
test_call()
is anUnassignedCall
.- call: synr.ast.Call
- class synr.ast.Var(span: synr.ast.Span, id: synr.ast.Id)
Bases:
synr.ast.Expr
A variable in an expression.
Examples
x
andy
inx = y
.x
inx + 2
.- id: synr.ast.Id
- static invalid() synr.ast.Var
- class synr.ast.While(span: synr.ast.Span, condition: synr.ast.Expr, body: synr.ast.Block)
Bases:
synr.ast.Stmt
An while statement.
Examples
while x <= 2: pass
Here
condition
isx <= 2
, andbody
will bepass
.- body: synr.ast.Block
- condition: synr.ast.Expr
- class synr.ast.With(span: synr.ast.Span, lhs: List[synr.ast.Var], rhs: synr.ast.Expr, body: synr.ast.Block)
Bases:
synr.ast.Stmt
A with statement.
Example
with open(x) as f: pass
Here
lhs
will bef
,rhs
will beopen(x)
, andbody
will bepass
.- body: synr.ast.Block
- lhs: List[synr.ast.Var]
- rhs: synr.ast.Expr
Error Handling
Synr uses a DiagnosticContext
to accumulate errors that occurred
during parsing. Multiple errors can be accumulated and they are all return
after parsing. DiagnosticContext
can be subclassed to customize
error handling behavior. We also provide a
PrinterDiagnosticContext
, which prints out errors as they occur.
- class synr.DiagnosticContext
- add_source(name: str, source: str) None
Add a file with source code to the context. This will be called before any call to
emit()
that contains a span in this file.
- emit(level: str, message: str, span: synr.ast.Span) None
Called when an error has occured.
- render() Optional[Any]
Render out all error messages. Can either return a value or raise and execption.
- class synr.PrinterDiagnosticContext(sources: Dict[str, Sequence[str]] = NOTHING, errors: List[Tuple[str, str, synr.ast.Span]] = NOTHING)
Bases:
synr.diagnostic_context.DiagnosticContext
Simple diagnostic context that prints the error and underlines its location in the source. Raises a RuntimeError on the first error hit.
Transforming the Synr AST to a Different Representation
to_ast()
allows you to transform a Synr AST into your desired
representation while using the existing Synr error handling infrastructure.
First implement a class that inherits from Transformer
, then
pass an instance of this class to to_ast()
. to_ast()
will
either return your converted class or an errors that occurred.
- class synr.Transformer(*args, **kwds)
A visitor to handle user specified transformations on the AST.
- do_transform(node: synr.ast.Node, diag: synr.diagnostic_context.DiagnosticContext) Optional[Union[synr.transformer.M, synr.transformer.F, synr.transformer.S, synr.transformer.E, synr.transformer.B, synr.transformer.P, synr.transformer.T]]
Entry point for the transformation.
This is called with the synr AST and the diagnostic context used in parsing the python AST.
- error(message, span)
Report an error on a given span.
- transform(node: synr.ast.Node) Optional[Union[synr.transformer.M, synr.transformer.F, synr.transformer.S, synr.transformer.E, synr.transformer.B, synr.transformer.P, synr.transformer.T]]
Visitor function.
Call this to recurse into child nodes.