Learning Julia
If you haven’t heard of Julia, it is a new programming language that advertises high performance (e.g. close to C or Fortran) while having a dynamic feel like R or Python. The promised performance gains is certainly the biggest draw card that Julia has compared to Python or R. The initial v1.0 was released quite recently (Aug 2018) and it is now up to v1.3 (released Nov 2019). One of the long-awaited features included in the latest release is automatic thread spawning (although it is marked as experimental). Syntactically it looks vaguely like a mix of Matlab and Python.
I’ve been picking up Julialang on and off for the past two years or so now, but never really got stuck into it. In recent weeks I’ve decided to take on two relatively big side projects. The first is to move the agricultural management model I developed as part of my PhD to Python 3, which I’ve named Agtor. The second was to port Agtor across to Julia. The reasoning for this is three-fold:
1) For various reasons Agtor is outgrowing the performance that Python can offer.
2) A clean Python implementation is still useful for demonstration and prototyping purposes (hence the port to Python 3 - not to mention Python 2 is end-of-life on 1 Jan 2020!)
3) I wanted to dive in and learn Julia with a non-trivial project
Some aspects of point 1 could be addressed by leveraging numba or Cython. From my admittedly brief experience with numba, its efficacy seems to be restricted to the computationally intensive - places where a lot of mathematical calculations occur. This is not really Agtor’s situation, as most of the runtime is attributable to calling C/Fortran libraries. A lot of the time is spent on loops and function/method calls.
Cython was used in the original project from which Agtor arose and as simple as it was to incorporate, and as great as the performance gains were, it could never replace all the Python code, and nor should it.
As with learning any new language there’s a lot to take in. I’m writing down some lessons learnt here, comparing against how things might be done in Python.
No “classical” class structure
To set the scene lets briefly go over object definition in Python. To create an object you would do something like:
class Cup(object):
def __init__(self, volume=0):
# volume in ml
self.volume = volume
Which defines a Cup
object and a constructor through which you can define its intial state (defaulting to empty: volume = 0
). You could use it like so:
my_cup = Cup(200)
Which would instantiate a Cup
filled with 200 ml of unknown liquid.
Lets extend the Cup with attributes and represent behaviour through methods. I, for some reason, also want some properties (in this case, dynamic setters and getters).
class Cup(object):
def __init__(self, volume=0, capacity=250):
self.volume = volume # volume in ml
self.capacity = capacity # max capacity
def drink(self, amount):
vol = self.volume
self.volume = max(vol - amount, 0)
def fill(self, amount):
vol = self.volume
cap = self.capacity
self.volume = min(vol + amount, cap)
@property
def volume_in_L(self):
"""Get volume in litres"""
return self.volume / 1000.0
@volume_in_L.setter
def volume_in_L(self, volume):
"""Set the cup to exact volume using litre values"""
vol_ml = volume * 1000.0
self.volume = min(vol_ml, self.capacity)
One more thing - I’d like to create other kinds of liquid vessels. Say, a mug:
class Mug(Cup):
def __init__(self, volume, capacity):
super().__init__(volume, capacity)
The above is of course class inheritance - the Mug inherits all the attributes, properties and methods as defined for the Cup
.
Now, how do we do this in Julia, given that it doesn’t support C++ style Object-Orientation.
-
Julia does not have classes (it has structs).
-
Structs do not support methods.
-
Julia does not support inheritance in the vein of C++.
Well, here’s a full code dump:
# The base type...
mutable struct Cup
volume
capacity
end
# functions tied to a type and its subtypes
function drink(c::Cup, amount)
c.volume = max(c.volume - amount, 0)
end
function fill(c::Cup, amount)
vol = self.volume
cap = self.capacity
c.volume = min(vol + amount, cap)
end
# to replicate class properties...
function Base.getproperty(c::Cup, v::Symbol)
if v == :volume_in_L
return c.volume / 1000.0
else
# Note we use getfield, instead of getproperty
# This is to avoid infinite recursion
return getfield(a, v)
end
end
# simulating setters
function Base.setproperty!(c::Cup, v::Symbol, value)
if v == :volume_in_L
vol_ml = value * 1000.0
c.volume = min(vol_ml, c.capacity)
else
# Note we use setfield, instead of setproperty
# This is to avoid infinite recursion
setfield!(c, v, value)
end
end
Simulating inheritance can be done like so:
mutable struct Mug <: Cup
volume
capacity
end
Because Mug
subtypes Cup
, all functions defined for Cup
will work for any Mug
object and indeed anything that is a subtype of a Cup
. This is the multiple dispatch system at work. Those of you who have worked with S3 and S4 classes in R will recognize this. Although working with the object oriented features of R is considered an advanced topic.
Here, we’ve simulated inheritance and object methods but note that we had to redefine the struct fields (volume
and capacity
) again. While this is explicit, in the real world it can become annoyingly verbose.
Thankfully there are alternatives available. The first I came across was to use the Mixers.jl
package. It provides a set of macros to help simulate inheritance.
The second is to use the macro system in Julia directly. I won’t claim I fully understand macros - it’s similar to macros in C, but as I currently understand it, different enough that the concepts are not completely transferrable.
Side-story: Macros
A workable, if naive, explanation is that wherever you see a macro being used it simply means “do a set of things here” - this “set of things” can include manipulating the code itself. This is called “metaprogramming” but I won’t be going into further detail here.
The very simple example below defines a macro @sayhello
. Wherever you use @sayhello
in your code gets replaced with a call to the println
function.
# Example taken from the docs:
# https://docs.julialang.org/en/v1/manual/metaprogramming/#man-macros-1
macro sayhello(name)
return :( println("Hello, ", $name) )
end
# @sayhello("Rick")
# > Hello, Rick
Well hang on, why bother with macros? We can do the same thing with a function with the same effect:
function sayhello(name)
println("Hello, ", $name)
end
The difference is in the parsing and execution of the code. Where normally code gets compiled then executed, macros allow for the ability to change the code before its compilation and execution. This allows for some stuff that gets… meta….
Back to the main story
Alright, so the gist of the above is that macros modify code before they get run. Below is the @def
macro which is now in fairly common usage.
macro def(name, definition)
return quote
macro $(esc(name))()
esc($(Expr(:quote, definition)))
end
end
end
What the @def
macro does is create another macro which inserts the specified code into where it is used. So if I state:
@def nevermore begin
println("Quoth the Raven")
end
# The compiler will replace `@nevermore` with the specified println call
Meta, right? But how does this help with inheritance?
Rather than use it as an on-the-fly search and replace, we can specify a common set of struct fields with defined types…
@def cup_fields begin
volume::Float64
capacity::Float64
has_handle::Bool
owner::String
end
As before, the newly specified macro (@cup_fields
) inserts the code defined within into wherever it is used.
Now, rather than typing out the fields for every subtype of Cup
, I can simply use the @cup_fields
macro.
mutable struct Cup
@cup_fields
end
mutable struct Mug <: Cup
@cup_fields
end
mutable struct ShotGlass <: Cup
@cup_fields
end
To make the example concrete, what Julia ends up parsing and running is the below.
# This:
# mutable struct ShotGlass <: Cup
# @cup_fields
# end
# Becomes:
mutable struct ShotGlass <: Cup
volume::Float64
capacity::Float64
has_handle::Bool
owner::String
end
Much less verbose!
That’s it for now, but check back later as I plan to write up a similar post on DataFrames. Before you go, you might want to check out…
Some other bits and pieces
Python | Julia | |
---|---|---|
Dict keys |
if key in some_dict:
print("The key was in the dict")
|
if haskey(some_dict, key)
print("The key was found!")
end
|
Argument unpacking |
def example(a, b, c):
print(a, b, c)
inputs = [1, 2, 3]
example(*inputs)
|
function example(a, b, c)
print(a, b, c)
end
inputs = [1 2 3]
example(inputs...)
|
Keyword argument unpacking |
def example(a, b, c):
print(a, b, c)
inputs = {a: 1, b: 2, c: 3}
example(**inputs)
|
# Note the semi-colon!
# In Julia, keyword arguments have to be explicitly
# declared. These are separated from regular
# positional arguments with the semi-colon.
function example(; a, b, c)
print(a, b, c)
end
inputs = Dict(
:a => 1,
:b => 2,
:c => 3
)
example(; inputs...)
|
Ternary operator | c = a if a > 0 else b |
c = a > 0 ? a : b |
String concatenation |
str_a = "Hello"
str_b = " World"
str_c = str_a + str_b
|
str_a = "Hello"
str_b = " World"
str_c = str_a * str_b
|
Iterating over a dictionary |
for k, v in dict:
pass
|
for (k, v) in dict
(k, v)
end
# In Julia: `k in dict` will produce
# the key->value
|