Archive for the ‘web’n’stuff’ Category
From the home page of the project:
We want to lower the barrier to entry, which is necessary to help everyday people start their first project, become makers and advance technology instead of using it. codebender requires no installation, so you can get started with Arduino programming the minute you get one in your hands! And with the development tools we provide, you can do so faster and easier! codebender also stores your code on the cloud, so it’s safe and accessible from anywhere, anytime.
Several nice features will be available soon, such as remote flashing: together with an Ethernet shield flashed with a properly designed TFTP bootloader, you will be able to upload a sketch remotely, over the internet! Another nice feature regards its integration with the open documentation available on the Arduino website, which will be accessible directly from the IDE by selecting a piece of code and, then, by pressing ctrl+space.
More information can be found here.
India is large … or should I say XXXL? In order to give a better support to all the Arduino interested people in India we have created a new section in our forum where to bring up issues in both English and Hindi. We know there is a growing community coming from schools, maker groups, and the hardware section of the free/open source movement.
To coordinate the efforts around Arduino in India, we have gathered an excellent group of forum moderators (their nicknames are mentioned for you to find them in the forum):
- Priya (our Arduino representative in India) – beachbrake
- Sudar - Sudar
- Nishant - NI$HANT
- Jobin – jo_v
(there is more to come, as we expect the Indian community to move quick, we will need all the help in the world)
So if you are in India and want to find your pals at our forum, refer to the following website and contribute!
India, our forum salutes you!
A device that opens the websites using physical world interaction? This is a step closer to the internet of things. Using our beloved Arduino and RFID tags, we can think of a lot many applications of this device!
Too lazy to create your own Arduino + RFID reader? No fear! Arduino internet Gizmo is here.
A detailed Step-by-step making instruction is given here.
What websites are you going to keep as your shortcuts today?
We have been bot’ed, it is kind of cool to show this image where we have 123.456 users in the forum, but it is not really true. We have experienced a pretty nicely made attack to the forum. Not that the machine slowed down, but we got zillions of new users … about 10x more users per day as we used. Check this graph:
We will erase all the 40.000 non-existing users from the forum as well as implement a whole lot of new patches to the software that will improve the work the moderators are making in the system. We are going to be working tomorrow, Friday 26th between 9.30AM and 11.00AM CET. You may experience small downtimes as we will be implementing incremental changes to the software making sure they work in production.
A weekend project by Adafruit to print announcements and discount coupons from the internet.
This project uses an Arduino Uno, an ethernet shield, a thermal printer (aka receipt printer) and a Staples easy button to print out the current discount percentages available at the tkts times square booth for Broadway shows.
A beautiful tutorial accompanies the project.
During the last year we have seen our community growing at a tremendous pace. This has its pros and cons. More people means more people to help out, but also more work in moderating the forum. We want to be inclusive at we work of supporting multi-language forum topics, as well as the multi-language IDE. The way this is done is through delegation and we have been welcoming new moderators recently who, by the way, are making a terrific job and are pushing the team in improving the Database’s response times with very creative solutions.
On top of that, we are opening the blog to guest bloggers. Davide Gomba, head of Officine Arduino Torino, is going to be coordinating the group of bloggers that will be posting contents to the Arduino Official site.
When it comes to the Spanish community, we have been looking at the work made by Arduteka (arduteka.com) in Zaragoza, Spain. They have been running a forum for some time and are very engaged in reaching people speaking Spanish from all around the world. Therefore we have decided to invite Pablo and his partner Carlos to become both moderators in our Forum and guest bloggers to write about projects made with Arduino as well as tutorials in Spanish and reviews of different Arduino-related products.
In this way we are not just enlarging our group of moderators, but bringing people with opinions and a strong interest in getting things done the right way. As Pablo and Carlos cannot be attending two forums at once, we will do our best to import the most informative of the messages in their current forum to also help their current user base feel more comfortable with the migration.
Here a quote from Arduteka’s announcement taken from their website (in Spanish):
Cuando llevábamos apenas dos meses, nos pedisteis la incorporación de un foro en que pudiéramos “conocernos” plantear proyectos, quedadas, dudas y demás…
De nuevo tuvo gran éxito entre vosotros incrementando aún más el flujo de visitas (hasta el punto que nuestro servidor esta cuasi saturado) y generando gran unión entre la comunidad, con una gran comunicación, llegando a diferentes países de habla hispana como España, Venezuela, México o Colombia
Pues bien amigos, todo este trabajo y esfuerzo no ha caído en saco roto para el equipo oficial de Arduino y a partir de ahora colaboraremos activamente en la difusión y organización de contenido en castellano en Arduino.cc!
El primer paso, será la unión de este foro con el oficial de Arduino.cc de manera que juntemos esfuerzos en la unión y difusión de la comunidad de habla hispana, uniremos experiencias y daremos forma al foro entre el moderador actual Coleoptero y yo.
Today we welcome three new forum moderators as well as the Dutch (Nederlands) board and a new structure for the French boards. JO3RI and JANTJE are our new Dutch moderators and Nick Gammon becomes part of our Global Moderator’s list.
We are experiencing a huge increase in the traffic on the Arduino site. Most of it goes to the forum. We are happy to see this development in the community. Last month was the first time we reached over 1.000.000 unique IPs on our server and this month we will reach it again. Here a graph showing the development of the visits to the Arduino website:
We are now working in preparing the next generation of the website. It is going to take some months before you see anything coming out, I can only say it will be great We want the Arduino site to be the best place for makers to meet up and share their projects … but it is going to demand a lot of effort to make it happen.
In the meanwhile we will keep on patching the server and we will make sure everything you guys are contributing with makes it to the next iteration of the system: forum messages, playground pages, and translations.
this is a very personal post to explain and send an apology to the community for our very sloppy server response over the last month or so. I will also rise a case for how the website works and for how we are envisioning our newcoming webservices.
I know some of you will appreciate this explanations, since there have been comments in the forum and requests for someone’s toes to be cut (probably mine). It is not that we haven’t been doing anything about it, we just didn’t communicate properly about what we were doing. Mostly because we were too busy getting it done. Therefore I apologize, I am a grown up bearded man with no problems saying I am sorry.
As many of you know, the Arduino project is growing quick. We are not experiencing the exponential growth we saw during 2010, but it respects a somehow lineal growth that makes my hair get a little grayer every time I look at the statistics on how our website is used. We are about to reach 1.000.000 unique visitors per month, adding up to over 53.000.000 hits on the website each month. At our previous server (because we upgraded last week) the 8 cores were 90% of the time at 100% of performance. Since we were renting a VPS server at ServInt, we could go over the use of the 8 cores. In other words, we could easily reach a peak of 9.5 virtual cores in weekdays.
This started to happen during late November 2011. Something unexpected since the end of the year has traditionally been a month when our stats would slow down until March, the time when student projects magically happen at very many education centers around the world. But, somehow, the deals made last year throughout the Arduino distribution network, brought much more traffic to the website and took me a little by surprise.
When I first noticed this issue, I took two decisions:
1) we need to make more periodic backups of the website, to assure we have everything there whenever we migrate the website
2) we need to upgrade the server asap
About the backups
I wrote a script to make daily backups of the website, it will put the DB down for a max of 10m early in the European morning. That is the message that some of our most committed forum participants experience when they go on answering questions at 6AM (man that is commitment).
You guys might come to the website at the time the script is backing up the whole database. In the future we will make this work differently. We are making a whole new backup server that will be making copies of everything while it happens and it will not stop the server of operating as it is now.
You might think … but ServInt is an internationally recognized provider, isn’t it giving you a proper backup tool? Our provider is in fact giving us a great backup system for free, but we want to have our own on top … you never know when it will get handy. We have 7 years of interaction in the forum and website that we don’t want to loose. Our knowledge base (our as in yours and ours) has been built collaboratively among the Arduino users since 2005 and we have the responsibility of keeping it alive as well as safe so that it doesn’t get lost.
About the upgrade
We have finally moved away from a VPS to our own machine!! This is exciting, the demand of the Arduino community is so high that we have been in the situation where we have to hire two machines in a data center to have the community up and running. It also means that we had to make a migration as we never did before.
I spent over a week discussing with ServInt how to get this done. I got promised they could make it in 30m. I did an upgrade all by myself early 2011 with only 5m downtime, so I thought they were being conservative in their approach. I like when people don’t try to lie to you to get you in, so I believed they could make our upgrade with 30m downtime.
I scheduled it for January 7th 4AM CET, a really good time, since it is the time we have the lowest traffic rate. Servint came in a day earlier informing they had run out of stock on the machine I wanted for us all to use. It took me a couple of hours to react to this and order the next machine in their line (more expensive, but more resources in it) what delayed the upgrade to happen on Sunday the 8th at 4AM CET, the second best time.
Unfortunately, the worst prediction from ServInt wasn’t as bad as reality. Our server was so badly overloaded that it took them over 20h to make the full migration. I assisted to three shift changes on the technicians side and I also witnessed how they accidentally turn off the server off during a 6h interval, when they shouldn’t have done it. This translated in many of you experiencing the server to be down when it shouldn’t. I got a formal apology from their side, we are good customers to them and they treated us nicely. I appreciated their honesty and took for granted they wouldn’t fail a second time. Despite the issues we faced, we have had a really good relationship to this provider over the last 3 years or so, and I am willing to keep it going as long as they keep the good work from now on.
About the Forum being dead slow
The forum makes use of a database. Every request you guys make, specific searches or posts or whatever, translates in a database query that uses CPU time. It is not about harddrive and it is not about memory … it is about CPU. All the requests made by you last the CPU, and the more visitors we get the more CPU time it consumes. Since we are still growing, the consumption of CPU power is our main concern, as everyone likes to interact with the forum. Correct me if I am wrong, but it is a helpful resource, both for looking for advice, but also for recreational purposes … people come, comment, joke, complaint … and we like it to be like that.
So the upgrade was necessary to keep up with the forum’s growth. But if something can go wrong, it might do so … and in this case it did. The forum’s database got a field modified what broke the Personal Messaging feature. At the same time, there were so many files uploaded in the forum’s folders, that we reached a protection quota I established about a year ago … both issues came at the very same time, probably with one day difference, but you experienced it as if it was the same source of problems. Of course, for the server to render that error would also take CPU time, slowing down the server’s reactions to your queries.
Both issues are now fixed and should not bother you for quite some time. The forum has about 100GB in uploads, personal pictures, etc. But the new machine has harddrive enough to handle this.
And now what?
Now, we will keep on working on things as usual. We will move the forum to a machine by itself at some point this year. We will implement database replication to totally cut away those downtimes in the European mornings and we will make sure the backups of the folders are taking as little CPU as possible using differential backup systems (that we aren’t using yet). I will work in bringing in some of the features many of you have requested for the forum over the last months and we will try to better integrate the different parts in the website with each other (yes, some more graphic design will help).
In the meanwhile I have to credit Cristian (aka Mr. Vacuum) for is very helpful contribution to the Forum and Identification system, and all of you for your patience during this last month or so. We will work in anticipating things much better from now on.
If you want to contribute to the Arduino website, you can use the Playground, which is an open wiki, but you can also be part of the Forum. You can help by volunteering to moderate a forum in your language (we can create boards in any language in the universe, just ask) or translating the Arduino reference to your mother tongue. Send an email to web [at] arduino DOT cc or comment on this board in the forum with your suggestions.
Yesterday we had a wonderful day at New York Hall of Science, for the Open Source Hardware Summit, in its second edition. We had the chance to see amazing presentations from makers all over the worlds [see schedule & list of participants] We had a good time in taking with Alicia Gibb, one of the organizers who explained us this year’s numbers and some ideas for the future.
[SlideShare presentation after the break]