2022 @phenomnominal

I'm Craig (he/him)

I do JavaScript.

2022 @phenomnominal

Desktop + Web Player

Slides!

2022 @phenomnominal

DISCLAIMER!

TRANS RIGHTS ARE HUMAN RIGHTS

Today, I'm going to tell you a story!

2022 @phenomnominal

But first...

✨MAGIC

SpEC attendee!

Good Guy

You're a wizard!

2022 @phenomnominal

You went to Hogwarts!!

(School of Witchcraft and Wizardry)

Place where stuff happens

2022 @phenomnominal

You can do magic!!

This is a patronus

2022 @phenomnominal

You know Dumbledore!!!

Good Guy

2022 @phenomnominal

And Harry Potter is your Best Friend!!!!

Main Good Guy

2022 @phenomnominal

But there's a problem...

Maybe
Good?
Maybe
 Bad?

2022 @phenomnominal

Voldemort is back (again!)

Literally the

Worst Guy

2022 @phenomnominal

And he has put a curse on Harry!

2022 @phenomnominal

2022 @phenomnominal

If we don't save him in the next 25 minutes, Voldemort is going to kill him!

2022 @phenomnominal

Not good.

Our friend is gone.

Time is running out.

2022 @phenomnominal

But there is one bit of good news...

2022 @phenomnominal

Harry speaks Parseltongue!

He can talk to snakes!

2022 @phenomnominal

Which, as every magical being knows, is a

Turing-complete programming language!

(this is 100% canon, don't look it up)

2022 @phenomnominal

Weakly Typed

Functional

Interpreted

Garbage collected

Significant whitespace

Parseltongue 🐍🐍🐍

2022 @phenomnominal

Parseltongue 🐍🐍🐍

Variable Declarations

Parseltongue:

JavaScript:

Only one type of variable

Always start with

Assignment Operator is

<~
sss

2022 @phenomnominal

Function Expressions

Parameters are defined with an array

Return uses the assignment operator

Significant whitespace

Parseltongue 🐍🐍🐍

Parseltongue:

JavaScript:

2022 @phenomnominal

Call Expression

Passing arguments to a function also uses the Assignment operator

Arguments are declared with an array

Parseltongue 🐍🐍🐍

Parseltongue:

JavaScript:

2022 @phenomnominal

Control Flow

Use an empty array when you have no arguments

Parseltongue 🐍🐍🐍

Parseltongue:

JavaScript:

2022 @phenomnominal

Loops

Range operator is

~>

Parseltongue 🐍🐍🐍

Parseltongue:

JavaScript:

2022 @phenomnominal

Spells

Parseltongue 🐍🐍🐍

Parseltongue:

JavaScript:

is magic and becomes

sssCast
alert

2022 @phenomnominal

2022 @phenomnominal

Harry's spell ⭐️⭐️⭐️

2022 @phenomnominal

Internet browsers don't understand Parseltongue.

2022 @phenomnominal

We need to translate Harry's Parseltongue into JavaScript!

2022 @phenomnominal

Parseltongue

JavaScript

???

2022 @phenomnominal

Parseltongue

JavaScript

REGEX!?

2022 @phenomnominal

Let's go a line at a time:

RegEx:

(the most evil magic)

2022 @phenomnominal

That gives us:

RegEx:

2022 @phenomnominal

Converting variables

RegEx:

2022 @phenomnominal

Converting loops

Hmmm...

RegEx:

2022 @phenomnominal

Converting functions

Ruh roh!

RegEx:

2022 @phenomnominal

2022 @phenomnominal

What are we really trying to do?

Transform one language into another

2022 @phenomnominal

Transpiling

(aka transfiguration)

2022 @phenomnominal

"Transpiling is a specific term for taking source code written in one language and transforming into another language that has a similar level of abstraction"

Transpiling

2022 @phenomnominal

Where do we start?

String literal

Function call

Identifier

Expression

2022 @phenomnominal

ESTree

2022 @phenomnominal

Esprima

2022 @phenomnominal

 Abstract

  disassociated from any specific instance

  the way in which linguistic elements are put together

 Syntax

 TREE

  data structure made up of vertices and edges without any cycles

2022 @phenomnominal

Wat.

2022 @phenomnominal

Wat.

2022 @phenomnominal

a data structure that represents the structure of code, without any actual syntax.

An AST is:

Code:

2022 @phenomnominal

a data structure that represents the structure of code, without any actual syntax.

An AST is:

AST:

2022 @phenomnominal

Which means...

2022 @phenomnominal

If we can get from Parseltongue to an AST...

Then we can go from that AST to JavaScript!

2022 @phenomnominal

Parseltongue

JavaScript

REGEX!?

2022 @phenomnominal

Parseltongue

JavaScript

AST

2022 @phenomnominal

Parseltongue

JavaScript

AST

???

???

2022 @phenomnominal

2022 @phenomnominal

Target AST

var spell = 'Expecto Patronum';

function magic (spell, intensity) {
    var intense = '';
    for (var i = 0; i < intensity, i = i + 1) {
        intense = intense + '!';
    }
    return spell + intense;
}

for (var i = 0; i < 10, i = i + 1) {
    magic(spell, i);
}

Remember Esprima?

2022 @phenomnominal

import { parse } from 'esprima'; 

const AST = parse(`
    var spell = 'Expecto Patronum';

    function magic (spell, intensity) {
        var intense = '';
        for (var i = 0; i < intensity, i = i + 1) {
            intense = intense + '!';
        }
        return spell + intense;
    }

    for (var i = 0; i < 10, i = i + 1) {
        magic(spell, i);
    }
`);

Remember Esprima?

Esprima takes code written in a specific language (JavaScript) and turns it into an AST!

2022 @phenomnominal

We need to make our own Esprima for Parseltongue!

2022 @phenomnominal

How do we do that?

2022 @phenomnominal

Parseltongue

JavaScript

AST

???

???

Tokens

Lexing

2022 @phenomnominal

Lexing/Tokenising

"Lexing is the process of breaking down source code into words that are relevant to the language, which are called tokens"

2022 @phenomnominal

Lexing/Tokenising

Tokens in JavaScript

Identifier

Keyword

Literal - bool, null, number, string

Punctuator

EOF

RegularExpression

Template

(we're getting a bit intense now - here's an owl)

2022 @phenomnominal

Lexing/Tokenising

Tokens in Parseltongue

Identifier

Keyword

Literal - bool, null, number, string

Punctuator

EOF

 

Indent

LineTerminator

Space

(we're getting a bit intense now - here's an owl)

2022 @phenomnominal

Identifier characters

Space

Punctuator characters

Space

Quote character

Lexing/Tokenising

Punctuator

Punctuator

String characters

Quote character

2022 @phenomnominal

2022 @phenomnominal

2022 @phenomnominal

Parseltongue

JavaScript

AST

Parsing

???

Tokens

Lexing

2022 @phenomnominal

Parsing

"Parsing is the process of taking the lexical tokens and applying the grammar of the language to them"

2022 @phenomnominal

Parsing

Variable Declaration

Identifier

Space

Punctuator - 

Space

String literal

<~

2022 @phenomnominal

Identifier

Space

Punctuator -

Identifier

Punctuator - 

Space

Identifier

Punctuator - 

Parsing

Function Expression

Line Terminator

Indent

[
,
]

2022 @phenomnominal

Parsing

Function Expression

2022 @phenomnominal

Grammar

Function Expression

2022 @phenomnominal

Variable Declaration

For Statement

Binary Expression

Update Expression

Actual LOL

2022 @phenomnominal

2022 @phenomnominal

Now we just need to go from the AST to JavaScript...

2022 @phenomnominal

Parseltongue

JavaScript

AST

Parsing

Code Generating

Tokens

Lexing

2022 @phenomnominal

Code Generating

This is really hard to do by ourselves.

We can again stand on the shoulders of half-giants!

2022 @phenomnominal

ESCodeGen

2022 @phenomnominal

2022 @phenomnominal

JavaScript!

2022 @phenomnominal

Parseltongue

JavaScript

AST

Parsing

Code Generating

Tokens

Lexing

2022 @phenomnominal

We're almost ready to save Harry!

2022 @phenomnominal

But OH NO!

Voldemort can also speak Parseltongue!

2022 @phenomnominal

Voldemort will try to stop Harry from escaping!

 

He might even use the...

2022 @phenomnominal

Unforgivable Functions

2022 @phenomnominal

We need to come up with a way to stop Voldemort from using those functions!

2022 @phenomnominal

JavaScript

AST

Parsing

Code Generating

Tokens

Lexing

Parseltongue

???

2022 @phenomnominal

JavaScript

AST

Parsing

Code Generating

Tokens

Lexing

Parseltongue

Inspecting

2022 @phenomnominal

Inspecting

Transformation

Mutation

Validation

2022 @phenomnominal

ESQuery

2022 @phenomnominal

Linting

Detect bad stuff

2022 @phenomnominal

Linting

Fixing bad stuff

Tickling curse

2022 @phenomnominal

2022 @phenomnominal

JavaScript

AST

Parsing

Code Generating

Tokens

Lexing

Parseltongue

Inspection

2022 @phenomnominal

2022 @phenomnominal

We did it!

Harry is safe!

2022 @phenomnominal

All is well

(again)

2022 @phenomnominal

So...

2022 @phenomnominal

When do you actually use an AST?

2022 @phenomnominal

Ever had to change every file in your codebase?

2022 @phenomnominal

Ever had to generate boilerplate?

2022 @phenomnominal

Ever had to extract API documentation?

2022 @phenomnominal

You can use an AST for that!

 

(and probably should)

2022 @phenomnominal

ASTs help us with...

Performance

Maintenance

Consistency

Analysis

Robustness

Automation

2022 @phenomnominal

They're everywhere

2022 @phenomnominal

They're powerful

2022 @phenomnominal

They're useful

2022 @phenomnominal

Once you know where to find them

2022 @phenomnominal

⚡️ Thanks! ⚡️

Fantastic ASTs and Where To Find Them (SpEC)

By Craig Spence

Fantastic ASTs and Where To Find Them (SpEC)

SpEC 2022

  • 2,525