Sunday, November 10, 2013

Brython vs PythonJS


If you have not heard of Brython, it is one of the most popular in-browser Python implementations. From the top of Brython's documentation: "Brython's goal is to replace Javascript with Python, as the scripting language for web browsers." Brython is a long way from reaching this goal, because as we will see below, it has serious performance issues. Brython is slow because of its conformant implementation of the Python standard, which is very dynamic and is generally hard to map to JavaScript.

The following tests compare the performance of Brython 1.1, Python 2.7, PyPy 1.9 and PythonJS 0.8.3. Note that PyPy tests were done with a warmed up JIT. These tests were performed with N=100,000 on a dual core 2.4GHZ machine with 4GB of ram. (test scores below are given in seconds)

Test1 - for loop:

 a = 0
 for x in range( N ):
  a += 1
  • Brython: 6.8249
  • Python: 0.0064
  • PyPy: 0.00267
  • PythonJS: 0.0009

In a simple for loop over a range, with a single variable incremented: Brython is 1,000 times slower than Python, and 7,500 times slower the PythonJS. Lets examine the output of Brython to see why it is so slow.

Brython Output

a=$globals["a"]=Number(0)
var $iter61=iter( range.__call__(N) )
var $no_break61=true;
while(true){
    try{
        x=$globals["x"]=$iter61.__next__()
    }
    catch($err){
      if($err.__name__=="StopIteration"){break}
      else{throw($err)}
    }
    document.$line_info=[21,"__main__"];None;
    $temp=Number(1)
    if(!hasattr(a,"__iadd__")){
        a=$globals["a"]=a.__add__($temp)
    }
    else{
        a=a.__iadd__($temp)
    }
}

As we can see above, Brython generates alot of code for such a simple for loop. Instead of using JavaScript's native number type, Brython has its own Number object. It has a try/catch to catch the StopIteration exception, and also it testing each loop if the number has a method __iadd__, and if not it falls back to using __add__. Lets take a look at how PythonJS translates the same code, and why it is 7,500 times faster in this case.

PythonJS Output

a = 0;
var x;
x = 0;
while(x < N) {
 a += 1
 x += 1
}

The PythonJS compiler knows that a for loop over a range is actually just a simple loop that can be translated to fast JavaScript.

Test2 - while loop:

 a = 0
 i = 0
 while i < N:
  a += 1
  i += 1
  • Brython: 13.5709
  • Python: 0.01008
  • PyPy: 0.00144
  • PythonJS: 0.001

Things get worse for Brython when the loop gets more complicated, this tests increments two values. In this test Brython is 1,300 times slower than Python.

Test3 - calling simple function:

 a = 0
 for x in range(N):
  a += no_args()
  • Brython: 7.717
  • Python: 0.0137
  • PyPy: 0.0025169
  • PythonJS: 0.0019

Test4 - function call with normal arguments:

 a = 0
 for x in range(N):
  a += no_kwargs(1,2,3)
  • Brython: 15.3209
  • Python: 0.0250
  • PyPy: 0.00279
  • PythonJS: 0.3729
  • PythonJS + with fastdef: 0.0039

In this test PythonJS is 14 times slower than Python, this is because by default PythonJS functions that take arguments, have logic to check those arguments at runtime, and to also check if the function was called from JavaScript and if so - adapt the arguments. This can be bypassed using the @fastdef function decorator, or fastdef with statement `with fastdef:`.

In the above test, PythonJS with fastdef is 6.4 times faster than Python. Note that you can still call fastdef function from external JavaScript, you only need to pack arguments into an array as the first argument, and pack keyword args into an Object as the second argument. Example: func( [x,y,z], {a:1, b:2, c:3} )

Test5 - function call with keyword arguments:

 a = 0
 for x in range(N):
  a += call_kwargs(1,2,3, x=1, y=2, z=3)
  • Brython: 14.75
  • Python: 0.0370
  • PyPy: 0.00417
  • PythonJS: 0.631
  • PythonJS + with fastdef: 0.006

Conclusion

Brython is slow and heavy, written almost entirely in JavaScript, and closely follows the Python language standards. Its runtime brython.js is 227KB. Its tokenizer and compiler, py2js.js is 3,500 lines of JavaScript code. It lacks static type analysis and optimizations. Brython is able to translate Python code to JavaScript on the client side. Its performance is much slower than Python.

PythonJS is fast and light weight, written in pure Python, and tries to balance performance concerns with the Python language standards (read more here). Its runtime pythonjs.js is 98KB. The PythonJS translator is 1,900 lines of Python code. The translator includes static type analysis, and optimizes operator overloading and attribute access. It lacks a tokenizer and currently can only translate code on the server side using Python2 as its host. Its performance can surpass Python and approach PyPy in some cases, because it tries to output JavaScript JIT friendly code.

PythonJS 0.8.3 Released

Today I have released PythonJS 0.8.3, this includes the new fastdef and fast for loop optimizations. The performance test above is also included under tests/test_performance.html. The next release of PythonJS will feature integration with NodeJS.

5 comments:

  1. You need a good test coverage for the code that is to be converted, and PythonJS also needs a test runner, because translating one language feature to another directly (vs Bpython interpretation as I got it) may have surprising results - https://mail.python.org/pipermail/python-ideas/2013-June/021035.html

    Language translation also sounds like a job for PyPy projects. I wonder what do you think about https://www.rfk.id.au/blog/entry/pypy-js-first-steps/ - is optimizing the JIT also possible from the other end? Might help Bpython.

    ReplyDelete
  2. Brython is not interpreting and executing line by line in the loop, the function is translated to JavaScript and then executed. PythonJS is doing the same thing as Brython, it translates the Python code, and then executes the JavaScript. The difference is that the code that Brython generates is not optimized compared to PythonJS, in these tests.

    I have looked at PyPy.js, but I do not think it is a practical solution to running Python in the Browser for the following reasons:
    1. it is 139MB
    2. asm.js is only fast in Firefox, and most people use Chrome.
    3. If you want to run your Python code in the Browser, all you have to do is translate it ahead of time with a translator like PythonJS or RapydScript. Running the PyPy interpreter client-side just to JIT the hot loops to JavaScript or asm.js is not a practical solution.

    ReplyDelete
  3. Thank for details. I miss the +1 button on this site. =)

    ReplyDelete
  4. @anatoly : I've been following closely the Ryan Kelly's posts on pypy.js . The concept is very promising but in the short term , it will be hard to get it working in the browser .

    @harts : you make a good point . thanks for taking the time to compare both and share your findings .

    ReplyDelete
  5. Brython speed has improved a lot since these tests were made, but it's still slower than PythonJS. It's the price to pay for compliance with Python

    Brython correctly runs this perfectly valid Python code (underscores replace indentation) :

    def range(x):
    ____return ['spam', 'eggs']

    for i in range(N):
    ____print(i)

    while the PythonJS compiler still thinks that the program iterates on the built-in function "range", and prints the integers from 0 to N...

    ReplyDelete