Arduino web architecture, back to the future
The last couple of months have been the most exciting of my entire career. Here at Arduino we are doing a big shift in terms of technology, user experience, and web improvements.
A new Arduino web ecosystem is arising, and the first hint of this new approach is visible in the Arduino Day website. This single-page website was the perfect fit to start experimenting with some new technologies we wanted to put in production. The Arduino Day website was a testbed for some new features of the new architecture we want to achieve.
Goals
- Website online and running 99.98% of the time
- Fast on every device
- Easy way to push updates online
- New theme with a single page app
- Clear division between presentation and functionalities (client/server architecture)
- Https whenever is possible
- Microservices
People
How did we achieve these results? It has been a challenge but, from November on, we started building a new Web team with a good mix of junior and more experienced developers. Putting together the team has been key, but we also needed to change some aspects of the way we work.
The team has been always under a tight schedule, at first we started fixing legacy bugs all around, many are still in progress, but we also started experimenting with new things. We started following the agile development methods, including a daily standup meeting to better coordinate our team of designers and developers, we create dedicated topic channel in our chat system to better tackle issues and follow the improvements roadmap, and we created clear todo lists organized by priority.
Technologies
- Go Language at the heart of our Web Applications
- Angular.js as the framework to create complex and nice websites
- PHP for widely used Application built by other communities (forum, blog, etc.)
We based most of our web applications on PHP for many reasons. It is widely used, there are many developers proficient in it, and many of the Arduino developers are familiar with it.
If you are a tech addicted like me you know there are many downsides of using PHP, or at least it was not a good fit for us for many different reasons.
We wanted to separate the frontend and the backend, we have frontend engineers and web designers that are very good at css/js, and backend engineers who knows how to design a REST API. The division of tasks allows developers to feel comfortable in their own expertise area.
We need frequent updates to our web applications and most of the apps are custom.
This ended up in having an easy way to update the system, with just a single binary and all dependencies in it. In addition we planned to have many microservices, each of them doing just one small thing, the Unix way. The language of our choice was Go.
Go is a relatively new language and allowed us to create REST API in few hours/days. It is impressively fast and soon we figured out that our bottleneck was not the language/framework neither the application, but our database (we are thinking about moving from MySQL to PostgreSQL, MongoDB or RethinkDB -very impressed by the last one-)
Having a single binary application made it extremely easy to setup a scalable architecture. Now adding a new server to our load balancer requires just a couple of minutes and it is almost completely automated.
The frontend of Arduino Day Website is written using the Angular.js framework, allowing us to have a very interactive and responsive interface with an easy way to deploy. Everything is deployed to a nginx server (soon we will use something like Amazon S3), the files are minified and compressed so that the website is relatively fast.
I know this is not exactly cutting edge, but it is very hard to introduce something new in an environment developed for many years and used by so many users of our community. You cannot just introduce a new feature or start a project from scratch, you need to support your old code, keep it running, maintaining it and possibly avoid downtimes and maintenance windows.
Learning from mistakes
We are makers, we made mistakes. Every transition is risky and could end up is some issues. We took the risk when we migrated the forum to a new unstable version and we made a mistake, we learnt a lot from it and we are trying to similar issues as much as possible in the future.
Doing good things is hard, it requires time and the constant feedback from our amazing community. Many of you helped us already giving feedback on the arduino.cc website, the Arduino Day website, the forum and so on. We keep improving the whole web platform as fast as we can, and obviously we are always looking for the right new talent to join the Arduino team.
April 16th, 2015 at 22:51:03
Did you consider Node.js instead of Go (it’s really interesting, too)?
If so, which were its downsides? Thanks
April 22nd, 2015 at 16:04:54
Yes we thought about it. The main advantage is to have a single binary and no dependencies + impressive performance compared to Node.js.
Node.js will be used in a new software we will release soon. If we will have time we will eventually rewrite it but need to see real effects on production. The rule for us is to keep good code as it is and write something new in golang.
April 22nd, 2015 at 21:40:25
You couldn’t have picked a better choice with Go. Congrats on a gophery future!