Teams at a low capacity lose their ability to contribute fast and well, morale will go down because they see less work being delivered, more reports of regression bugs and the feeling that there's no time to work on ideal solutions. In the end, most of the work is to fulfill the pressure of the business.

In the book "The elegant puzzle", Will Larson introduces the four stages of a team and what they need to be able to increase performance. He shares that for teams that are falling behind, the fix is to hire more people while providing tactical support, which is a logical step given the low stock of talent. Furthermore, he emphasizes that changing a team from one state to the other is a slow process, which demands patience and a solid reaction from the manager.

I've been working for the past 6 years in startups and this scenario is very familiar to me, including situations where not even hiring was an option. I would like to share what's on my playbook to keep a team stable. And I hope these tips help your teams as well.

Adapt the workflow to a new reality

The first hard truth to swallow is that your team won't be able to handle too many parallel initiatives as before. The reason being that working in simultaneous different tasks demands switching contexts frequently. This will, as a consequence, increase the stress in the development team and reflect poorly on the code being released to customers.

They also will be taking longer to complete their tasks because you don't have the collaborative environment you had before. No pair programming, not enough peers to review code, less people to share ideas and discuss problems.

Find the balance by making your development process sustainable again:

• Tame the backlog: the tendency is that it will only grow. Backlogs always get messy with time anyway and you should review it and remove any tasks that are not a top priority. Don't underestimate the power of simply hiding it from the team and keeping tasks in a backlog of the backlog.
• Start less, finish more: finishing a feature brings more value to customers than starting new ones. Make sure every epic or initiative that's started has an end before picking the next big thing. Even if to make it happen we concentrate the whole team on it. This will help to decrease context switching.
• Shutdown external requests channels: I'm all in for software engineers collaborating with customers and business teams, but embrace the reality that this can drain their focus now. It might be a good moment to put someone (or yourself) in between representing the team and optimizing their time.
• Never trade off quality over speed: it can be contradictory, but you need to empower your team to accept the lower speed of delivery. It's easy to sacrifice quality over speed in these situations, but the pay back is expensive and it comes in the form of regression bugs and unexpected fires to put out. It's better to do it right, instead of faster.

These tips are targeted to help us to deal with the work in progress, but remember that the backlog will continue to grow. Let's see how we can take care of that.

Focus!

Given that hiring is a slow process, the only option we have to avoid the team burning out is to narrow the focus of work.

Break it down, work on smaller things. Developing a feature with large scope and complexity increases the chance that the team will spend a long time on it. It can steal time that would be used to solve tech debts or even fix a critical issue in production. Work with the product team to create smaller iterations that can be released as early as possible to users. As a consequence, your team will feel that they are always progressing and delivering often.

This exercise can also be applied to tasks. Small tasks are easy to code, review and release. It will pump up the cycle time and throughput metrics.

It might be good to even stop feature development at all and focus only on tech debts. It is hard to negotiate with the product team, but worth a try, especially if you can demonstrate that fixing them will improve users happiness more than any other new feature they propose.

Be rigorous in the refinement process. When a cross-functional team is working together for a long time, the specifications tend to omit details and cover only the surface of the requirements. It's understandable, teams involved trust each other and the fine details are sorted out during development. Now, with low capacity, people are not as available as before and clarifying topics will take longer.

I'm assuming that at least product research and discovery is done well, if that's not the case, challenge every product assumption that's not backed by irrefutable evidence. And of course don't turn this into a petty revenge.

Being more criterious over the specification input will shift the workload back to the product team, decreasing the overhead of communication once the feature is started. This can even have a positive effect on the test results and numbers of regressions.

Avoid committing to deadlines or estimations. The reason is that now there's too many uncontrollable variables influencing how long a task will take. With fewer people to work on the project, every interruption, incident or meeting will have a greater influence on the deadline compared to a team with an adequate size. If a company wants to get serious about deadlines, then get serious about building great teams first.

Estimating well is an important skill to have, don't get me wrong, especially when the deadline is defined by an external factor. You should still make the effort to have at least a baseline to share with your team and compare with the final result. Then use these outcomes to negotiate scope of the next features.

Organize special activities. We call it Hackday, in your company you might use a different name for it. Software engineers have a whole day to work on whatever project they want. It may sound counterintuitive, but allowing them to relax from the daily pressure and work on their favorite initiatives, develop themselves and accomplish personal career goals is a great way to keep a good cadence during a crisis.

In more than 10 years doing these hackdays, I never had the feeling that I was wasting company money or time. There were always good outcomes from it, being a fix to an old tech debt or bringing joy and satisfaction to the teams.

Care about people

The level of stress will increase when teams have a shortage of talents. People will start to question themselves whether they should find some other place to work. It's your responsibility as a manager to know how this situation is affecting the team and help them to overcome the problems.

Ensure you have a psychologically safe environment. There's a good chance that failures will happen more often now. It's your job to encourage your team to speak up about their ideas, concerns and failures without fearing retaliation or bad judgment. This behavior will promote more trust between the team and also increase transparency in the environment.

Be honest. The worst that you can do is to try to hide from the team the reality when the side effects are so perceptible. Sharing the problem with them opens the door to discussions that can lead to solutions, put them in control of the situation and promote an inclusive environment. In addition, be calm and be pragmatic about the actions you take because if the team perceive you as impatient, irritated or desperate then you can transfer these emotions to them.

Pay attention to burnout. Keep an eye on whether people are overworking, being invited to too many meetings, getting interrupted a lot during work or getting requests outside of work hours. Use 1:1 meetings well so you and them can be open about the main issues that are leading them to stress out.

If you see strong signs of burnout, act to improve their work life on whatever is in your power and advise them to look for external help if necessary.

If you plan well, you can decrease the impact of a person leaving. The transition process will be smooth if you have a clear plan. You should look to hire a replacement, define how the communication to the team will be done and how the handover will be.

Setting expectations

I have been so far sharing tips on how to take some pressure off the team and create a sustainable environment. However, don't forget that you are running a business. While it's inevitable that there will be an impact on performance, it's crucial to stay in tune with business demands and set clear expectations.

In a healthy organization we expect a good collaboration among all the cross-functional teams to solve the low capacity problem, but let's not be naive, they will come for you once the releases are getting late or customers are complaining.

Match the scope of your team's ownership to the people available. I do that with a table:

• Columns are all components of the product the team takes care;
• Rows are team members;
• Values are the percentage of rough estimation of allocation of a given person, a row of values should always sum to 100%, because no overwork;
• The last row is a sum of values in the column, representing how much allocation a given component usually has;
• If you are negotiating headcount, add another row to indicate what's the ideal number of people in a given topic.

This approach makes it very easy to visualize which components we are ignoring due to having fewer people in the team. In my experience this table always helped business teams and top management to understand the situation. In addition, it's a great tool to bring to their awareness that software development is not only about building new features, but it requires maintenance, quality and infrastructure management.

Have your metrics in hand. The allocation table, despite being helpful, is a bit subjective. Nowadays it is not hard to extract metrics from the software development tools we use. The key takeaway is to have a baseline to compare previous performance with the current one.

• Cycle time, the time from when the first commit is done till the task is released in production, tends to increase. Applies also to Pull Request merge time or any duration-based metric;
• Throughput or number of deploys/releases tend to get lower. Burndown charts get flatter;
• The size of the backlog increases. You can also observe task columns in the board having more tickets than normal;
• Ratio of failed builds or regressions also tend to increase;
• If you have any product or business metric that's impacted by the low capacity, it can make even a better case for investing in the team.

Celebrate wins. I mean, this should be done independently of having a team falling behind or not. But any injection of happiness and optimism is always good to keep them motivated. Plus it will show to the rest of the company how valuable your team contributions are.

Now what?

Could it happen that by trying all these tips my team starts to perform better and better to the point that it's not necessary to hire anymore?

The answer is... No.

Just like the Covid vaccine won't be protecting you 100% from infection, but will definitely increase your chances of having just mild symptoms and surviving. These tips will help your team to go through a tough time with a solid foundation to rely on.

Nevertheless, don't fool yourself. Hire as soon as you can! There's always a limit for patience and it will be a bumpy road. I myself collect some successes but also failures trying to keep a falling behind team together.

A short appendix:

Is your company trying to shut down your project or department? It can be pretty unpleasant to realize this when you're the non informed.

In that case the tips in this article won't be that helpful. I recommend planning a transition with your manager and helping your team to do the same, being in the same company or outside of it.

Thanks to Luiz Rocha for reviewing this post.

In the first weeks with a 3D printer you usually rely on pre-designed models in the several 3D model repositories available for free, but it's inevitable that at some point you will want to design your own models. And among the several tools available, I chose Blender to learn and use.

Blender is an ultra complete, professional, free and open source 3D creation suite. I personally like the combo made by free software, big community support and a super professional tool. In case I get myself too involved at least I don't need to move to a new tool, because Blender has no limits!

I'll show you how to setup a boilerplate Blender file that has everything you need to start 3D modeling and avoid the common pitfalls that can affect your print later. This article won't teach you how to model or how to get your model in the printer, but I'll give some tips and directions on how to get there.

Learning Blender

First you need to install it: www.blender.org/download/. For this post I'm using Blender 2.90.1, but even if you have a newer version you should be able to follow it.

Being the popular tool that Blender is, there's plenty of online resources to learn it. I'll save your time and recommend the one I watched:

Ram Singh is the author of this video series where he teaches Blender in 7 days. You can easily see how dexterous he is in the tool after a few minutes watching it and I personally liked the pace of the course which is not slow and you can learn a lot in a short period of time. For 3D printing you will need at least to watch the series until day 3, covering the basics, editing objects and modifiers.

3D Printing setup

From now on, if you wish, you can open Blender and follow along the setup of this boilerplate file. Feel free to save the file to use as a template in the future. You can also override the default startup file with File > Defaults > Save Startup File.

1. Cleaning

When you start Blender, you get the following screen:

The first step is to get rid of elements we don't need in 3D printing, which are the camera and light. To delete them, just select them in the right sidebar, the one with the Scene Collection, and press X while selecting camera and light or use the context menu.

2. What you see should be what you will get

3D printers are precise instruments and knowing the exact size of objects when modeling is crucial to have a printed object in the dimensions you planned. With an object selected in Blender, if you press N you can see the dimensions of the object, presented in the image below.

The cube in the image has an edge of 2 meters and the grid where the cube lies has 1 meter of distance between each line. While I wished we had a printer big enough to print objects like that, we need to scale down the units to be able to handle smaller objects and we also need a grid compatible with our printer capable volume.

Units

To have better units to work on your model, open the Scene window as the image shows below and change the following settings:

• Unit scale: from 1 to 0.001. It defines the scale factor to use when converting between internal units and values displayed in the user interface.
• Length: from meters to millimeters

If you see the object dimensions above, you will realize that now it shows as having an edge of 2 millimeters instead of 2 meters.

Grid

The grid is still showing 1 meter between each line. Having a properly sized grid is very important because it helps us to quickly check sizes when modeling and also to align different objects. Ideally the grid should have a 1 millimeter granularity.

To set a new grid size you click in the Overlays option as the image shows below and change the scale of the grid to 0.001.

Now the grid in the image is showing a 1 millimeter granularity.

Working Area

Next step is to make visible the boundaries you should be constrained so the object fits in your 3D printer range. This step is optional because usually you can change the object scale in the Slicer software (the tool that converts the 3D object in instructions for your printer), but if you want to have that control, do the following:

1. Select the cube with mouse right-button click
2. Set its dimensions to the dimensions of your 3D printer (press N to see the Item tab)
3. Set its location to x = 0, y = 0 and z = height/2 (z is actually in the center of mass, not in the base, that's why it's not zero)
4. Click on the Object properties tab as indicated in the image below
5. In the "Visibility" untick Selectable
6. In the "Viewport Display" tab, choose "Display As" to Bounds

Then you should see the Working Area as the image below. I renamed the object name to "Working Area" so you don't mix with other objects in the collection.

3. Getting the details right

Edge measurement

It's common to have parts of your model that you need to ensure precise measurements, such as holes, gaps, parts that will be attached to others, like a screw, for example. To help you to know exact edge measurements, Blender has an option in the Viewport Overlay, as shown below:

Once enabled, it will show the measurement of the selected edges. There's an important catch, if you scale an object, this measurement might be "wrong" because it only gets updated if you Apply the scale transformation by selecting the object and pressing CMD + A or Ctrl + A and choosing "Scale" transform to be applied.

3D print add-on

Finally, there's an essential Blender add-on that helps us to check if the object we will be exporting is "healthy", i.e. doesn't have any problem like being non-manifold, super thin surfaces, too sharp edges or face intersections. The great thing about this add-on is that it can fix your model with a single click in most of the cases.

To enable it go to Edit > Preferences, then select Add-ons, search for "3D-print" and enable the add-on. You now have a side tab when you press N named "3D-Print". Then with an object selected, try to click the "Check All" button to see the results.

Instead of explaining the details of this add-on, I recommend you to explore the features with a cube object or try to "break" your object to see how the add-on reports the problems.

Once your object passes all checks, you can use the Export button to save it in the desired format for processing in the Slicer software.

Now you build

That's it! With these steps you will be able to start building your object ensuring that all edges are in the size you want and that your model is printable without issues. We just scratched the surface of Blender and there's still a lot to learn, but I can attest that this setup helped me a lot since I started to play with 3D modeling.

Here's a magnetic key holder I designed and printed to use at home:

As a suggestion for further reading, these topics might interest you and can help to avoid other pitfalls:

• Learn how to use the Boolean modifier or use BoolTool add-on.
• Learn how to use Mesh Analysis.

However, it can be annoying when you depend on productivity tools that are not available on all platforms. You also start to think twice before buying a license for a tool that won't be available on one of the platforms you adopt. Cross platform consistent user experience is hard, but we can find ways to solve this issue. This article will focus on how to improve the experience for one kind of tool:

Keyboard triggered Productivity and Assistant tool

The title above is my tentative to classify tools like Mac Alfred, Ubuntu Unity Dash, Gnome Shell overview and others.

Above a screenshot of Alfred in action.

The way they work is simple. After a global keyboard shortcut is hit by the user, they can type commands to:

• Launch applications
• Search desktop
• Open URLs and search specific sites
• Calculate something
• Manage clipboard
• Even trigger more complex workflows that can be customized

It's a true Swiss-army tool that saves a lot of time, but given that most of them are not cross-platform because of their deep relationship with the operating system, it's really hard to find a consistent experience. If we had a tool that was as powerful as these ones, reachable from a simple keyboard shortcut and available on all these platforms would be nice, right? Right?

Introducing… Terminal

Nothing new here, just the plain and simple terminal you are used to have available in any operating system. Basically we just need to find a way to have the same experience as these keyboard triggered productivity and assistant tools. As I'll show below, it's totally possible to achieve something similar, sometimes even more powerful. Let's do it!

Keyboard Shortcut

First important thing is to have a terminal that pops up by hitting a simple keyboard shortcut, luckily this is a feature that can be easily found in the most popular Terminal apps.

On Mac, I use iTerm2 support for Hotkey window and CMD+space to trigger that terminal.

On Linux, I use Guake with Super+space shortcut.

Usually I configure these terminals to hide when losing focus so you can have a better experience when executing a command that opens the browser, for example. You can also tweak the terminal fonts and color the way you want.

Above: Guake terminal screen on ArchLinux with a Timezone custom command I created.

So far so good, we have a quick way of opening a Terminal, now we need a good set of commands and packages that reproduces the nice behavior we get with the other tools.

Unix Shell

On both Mac and Linux I use zsh and oh-my-zsh and this combination is perfect because already comes with a pretty good way of organizing the features I want to have on my productivity tool set. By creating a oh-my-zsh plugin you keep all configurations in one place and can easily edit them to tweak existing or add new features.

I have this file on my oh-my-zsh install (with a quite suggestive name):

$ ~/.oh-my-zsh/custom/plugins/terminalfred/terminalfred.plugin.zsh

All aliases and functions you add to that file will be available in your PATH when you open the terminal. In the next sections I'll show what I have on this file. If you are not patient, you can take a look on this Gist.

What I also do is to keep that file on Dropbox and symlink to it from all machines I use. It's okay if you don't have any sensitive information in its content.

I'm confident that you can find a similar solution no matter the Unix Shell you use because in the end, what you need is only installed packages, aliases and functions.

Searching the web

This is the most used feature and oh-my-zsh has a perfect plugin, called web-search, that allows me to do this:

$ ddg how to ride a bicycle   # search duckduckgo
$ image beautiful bicycle     # search for images
$ map bicycle store           # search on a map

These commands will open a browser and do the search, more straightforward impossible. If you want to add your own custom searches, my approach to that was to copy that function in my custom plugin and add them manually:

alias urban=’terminalfred urban’
alias regex=’terminalfred regex’
alias shell=’terminalfred shell’
alias jsonb=’terminalfred jsonb’
alias aur=’terminalfred aur’

function terminalfred() {
	emulate -L zsh

	# define search engine URLS
	typeset -A urls
	urls=(
		urban "http://www.urbandictionary.com/define.php?term="
		regex "https://regex101.com/library?orderBy=RELEVANCE&search="
		shell "http://explainshell.com/explain?cmd="
		jsonb "http://codebeautify.org/jsonviewer"
		aur "https://www.archlinux.org/packages/?q="
	)

	if [[ -z "$urls[$1]" ]]; then
		echo "Check available searches"
		return 1
	fi
	if [[ $# -gt 1 ]]; then
		url="${urls[$1]}${(j:+:)@[2,-1]}"
	else
		url="${(j://:)${(s:/:)urls[$1]}[1,2]}"
	fi
	open_command "$url"
}

Quick calculator

From time to time I need to do quick calculations, for that one I use Calc — C-style arbitrary precision calculator:

$ calc 24 * 4
     96
$ calc
C-style arbitrary precision calculator (version 2.12.5.0)
Calc is open software. For license details type:  help copyright
[Type "exit" to exit, or "help" for help.]

; 365/12
 ~30.41666666666666666667
; .*12
 365
; exit

Dictionary

Few people know DICT, a dictionary network protocol and its command line companion dictd, a dictionary database server. After installing dictionaries, it's fairly simple to use and the result is formidable:

$ dict formidable
3 definitions found

From WordNet (r) 3.0 (2006) [wn]:

formidable
      adj 1: extremely impressive in strength or excellence; "a
             formidable opponent"; "the challenge was formidable";
             "had a formidable array of compositions to his credit";
             "the formidable army of brains at the Prime Minister's
             disposal"
      2: inspiring fear; "the formidable prospect of major surgery";
         "a tougher and more redoubtable adversary than the heel-
         clicking, jackbooted fanatic"- G.H.Johnston; "something
         unnerving and prisonlike about high grey wall" [syn:
         {formidable}, {redoubtable}, {unnerving}]
...

To make it easy to use a few dictionaries and make the result more brief, I have the following aliases:

$ alias | grep dict
define='dict -d gcide'          # english definitions
dicts='dict -d moby-thesaurus'  # synonym dictionary

Follow these instructions if you want to install it.

Translations

Being a Brazilian living in Berlin makes me a heavy user of translation services. Translate Shell is the best tool for that.

$ trans -brief ‘Super fácil traduzir com essa ferramenta!’
Super easy to translate with this tool!

Of course having some alias to make commands shorter is always good:

$ alias | grep trans
tre=’trans -b’                 # translate with brief output
trep=’trans -b :pt’            # translate to Portuguese
tres=’trans -shell -brief’     # open a REPL to translate
trev=’trans -v’                # view translation in terminal pager

Task management

I use Trello as task manager and I have a simple board to track my daily TODOs, tasks that I should be focusing during the day. Another wanted featured was to be able to add tasks easily to that board.

Above: Sample execution, now on Mac.

Implementation is done using the API. I could have installed a Trello command line tool, but I decided to keep it simple and I have the following:

alias tasks='curl -s -X GET -H "Cache-Control: no-cache" -H "Terminalfred" "https://api.trello.com/1/lists/569623d6blablabla309c4d301?cards=open&card_fields=name&fields=cards&key=`cat ~/.secrets/trello.key.secret`&token=`cat ~/.secrets/trello.token.secret`" | jq ".cards[].name"'

function taskadd {
    str="$*"
    if [ -z "$str" ]; then
        echo usage: $0 task
    else
        curl -s -X POST -H "Cache-Control: no-cache" --data-urlencode "name=${str}" -H "Terminalfred" "https://api.trello.com/1/cards/?idList=569623d660blablablad301&key=`cat ~/.secrets/trello.key.secret`&token=`cat ~/.secrets/trello.token.secret`" | jq ".url"
    fi
}

In the code above I have one alias that basically do a request to fetch cards from a specific list on a board of mine. The result is in JSON format, then I use jq tool, a json command line processor to have a nice output like the snapshot shows above. The function to add a task is basically doing another request to create a card on the same list and as a result, I just add the link on the response.

Given that I want to sync zsh plugin file on Dropbox, the code expects to fetch Trello API secrets from a specific folder in my machines with the right permissions. Be safe!

Launching applications

To mention one limitation of using exclusively the terminal, I didn't find a good way to replace GUI application launch and sometimes I still rely on Alfred, Gnome Dash or simply global shortcuts to open them. Even so, I have found one function in a StackOverflow thread that works if the GUI app command is available on the PATH:

o() {
    if [ $# -gt 0 ] ; then
        # Expand if $1 is an alias
        if [ $(alias | awk -F "[ =]" '{print $2}' | grep -x $1) > 0 ] ; then
            set -- $(alias $1 | awk -F "['']" '{print $2}') "${@:2}"
        fi
        ($@ &) &>/dev/null
    else
        echo "missing argument"
    fi
}

The main benefit is that you start the GUI app detached from terminal and avoiding stdout flooding your shell. However, to be honest, I don't use that much.

Everything is possible

If you achieved that point in the article I believe you get the point, we just scratched the surface of what can be done with a Terminal and basically there's no limit on what you can do to automate tasks and be more productive. You can find other tools I use, such as consulting timezones or finding files, in my plugin file.

Hope the lives of cross platform users are improved a little bit with that approach. As a last note, I purposely ignored Windows because I'm not an active user, but if you are able to have a similar setup using it let me know and I can update this article or link to yours.

This post was originally published on Hackernoon.

I partnered again with Luiz Rocha, a long time friend, to build this playground using Scrapy and Solr.

The code is available at Github

The project is structured in X parts:

• Web scraper that fetches the content of all bookmarked links in Pinboard.in, using Scrapy.
• Index that is stored in Solr search engine
• A search frontend built in python Flask framework that work as a Solr client.

The project README shows you how to setup it yourself so feel free to use it in case you need to play around search engines and web scraping.

Twitter Flock event in Seoul, South Korea

Beyond doing talks in several events, I also built some sample projects that were used as canonical examples of the platform or simply to show off the platform capabilities.

Live Event Android App

A sample Android Live Event app using Fabric to help your participants to follow what's happening around an event at Twitter.

Source code available in Github.

Cannonball Android App

In October 2014 Twitter launched Fabric, a platform for mobile developers that provides several tools to improve their mobile apps. To showcase the tools this platform provided we built Cannonball demo app, a word game similar to magnetic poetry, in which the user can create poems, share with friends, see the popular ones all using the platform features such as: digits, social login, twitter kit, mopub, crashlytics and answers.

Source code available in Github.

This is the result after a few weeks of work:

Architecture

Under the hood this is what's happening:

                +---------------------------------+
                |                                 |
Tweet           | Raspberry Pi                    |
(music request) |                                 |
   +            | +-----------+      +----------+ |       Music powered by
   |            | | Twitter   | +--> | Mopidy   | |             Spotify
   +--------------> Streaming |      | Server   +------------>  Youtube
     network    | | API       | <--+ |          | |  audio      Soundcloud
                | +-----------+      +----------+ |
                |                                 |
                +---------------------------------+

This is what's running on the Raspberry Pi:

• Mopidy python open source music server
• Mopidy Spotify, Youtube, Soundcloud plugins that sources audio from these services.
• Written in Node.js
• UsesTwitter REST and Streaming API to monitor mentions to the bot username requesting music.

Twitter Bot syntax

The syntax is based on a simple rule:

@jukebox_user song name [by artist] [#spotify|#youtube|#soundcloud]

or

@jukebox_user [link to a spotify, youtube or soundcloud music]

• bot will play the first result found for the query sent to Mopidy
• artist is optional, but can help to search the right song
• you can restrict the search to a specific source by using the hashtag with source name
• if no music #source is specified, it will play the first result found on Spotify, then Youtube, then Soundcloud

A pre-configured admin user can send DM messages to jukebox user to #play, #pause, #next (skip song).

Hardware

To be able to run out of the box, I improvised a case with a display and some buttons to control the music, the skip button is especially important to avoid some unwanted songs 😁.

The schematic below shows the connections from raspberry PI GPIO and the buttons and displays.

Source Code

You can find the source code in Github. Hope you have as much fun as I did running this jukebox in the office.

Sometimes a good API documentation or playground is enough to understand and grow adoption of a platform. To help developers to cope very easily with an API, providing integration code in the language they are familiar is a good way to drive a faster adoption, because it will be clear how to use that API on code and not simply a bunch of isolated requests and responses.

Rosetta Code is an website that allow you to compare certain algorithm implementations in several languages. Inspired on that, API Rosetta Code goal is to allow developers to have certain use cases of the API implemented in several languages and libraries.

By simply following a directory organization and running a generator, you can have a website with all the samples built by your software engineers or by the community. To launch this experiment on Twitter, I organized 3 coding dojos to estimulate contributions from the developer community in Brazil.

If you want to know how it works to use in your platform ecosystem, check the project source code in Github.

Billions of fans around the world are tuned in to the World Cup matches, and many use Twitter to support their teams and join in the global public conversation about the games.

At the end of May, we held a World Cup-themed hackathon in São Paulo, Brazil. Between helping to organize the event and mentoring some of the teams, I played a little bit with Raspberry Pi and the Twitter Streaming API. My idea was to build a goal bell that would ring when the volume of Tweets around a specific goal grew very quickly. In order to do this, I had to connect to the Streaming API so as to calculate the frequency of Tweets mentioning “goal”, and create a device that would ring a bell when a spike in that frequency was detected.

Spoiler alert: if you are curious about how I built the bell before hearing the specifics, take a look at this video:

Detecting spikes in Tweet volume

Detecting relevant events in time-series data is not exactly rocket science, and there are plenty of research articles on that topic. But sometimes the implementation of these techniques is too complex to be applied in short period of time. I decided to build a simpler, less-precise solution that would work well enough for my needs. I started with the hypothesis that during any given game, lots of fans would Tweet when a goal was scored.

Thankfully, some simulations I did demonstrated that this pulse can be easily detected. Take a look at this data from the second half of the first World Cup match between Brazil and Croatia:

Image 1: Tweets per second mentioning “gol copa, gool copa, goool, golaço” during the match June 12th, 2014.

To get this data, I simply connected to the POST status/filter endpoint of the Streaming API. There I monitored for mentions of the Brazilian Portuguese words for “goal”, “world cup” and their variations, and then calculated their frequency in Tweets per second (TPS). Looking at the resulting graph above it is easy to identify when the goals occured.

The next step was to convert this noisy data into a baseline value that could serve as a trigger for the bell when that threshold was reached. To achieve this, I used a simple statistical tool: exponential moving average (EMA). Rather than showing you all the formulas, let’s instead visualize what EMA does:

Image 2: EMA calculated based on Tweets per second around the first goal during the second half of Brazil vs. Croatia.

As you can see, EMA smoothed the noisy Tweets-per-second data and gave us a cleaner time-series data set. To detect the goal spike, I then calculated a simple growth value relative to 10 seconds prior: (EMA now - EMA 10 sec ago) / EMA 10 sec ago. Let’s again use a visualization to see how this formula helped to determine the baseline trigger:

Image 3: Spike detection using relative growth around the first goal during the second half of Brazil vs. Croatia.

When the growth calculation results were greater than 1.5 (which meant that we saw a 150% growth rate in Tweets per second between present and the previous 10 seconds) the algorithm fired the trigger that happily rang the goal bell. Let’s zoom out and see the whole second half, removing the EMA curve:

Image 4: Spike detection using relative growth during the second half of Brazil vs. Croatia. Yes, we had two goals scored during this period.

It was interesting to see that the first goal had stronger growth. In my opinion, Twitter users were ready to tweet as soon as the goal was scored (because it was a penalty kick by Neymar), which resulted in more people tweeting over a shorter period of time. As you can also see on the last graph, the way growth was calculated helped to identify only when the Tweets-per-second frequency varied more quickly and in a larger amplitude (i.e., when a very exciting event happened during a game).

There are implementations of these calculations in Python and Ruby if you want to try them out yourself. The code for connecting to the Streaming API is relatively short and simple:

@client = TweetStream::Client.new

@client.on_error do |message|
  puts "ERROR: #{message}"
end
@client.on_enhance_your_calm do
  puts "Calm down"
end
@client.on_limit do |skip_count|
  puts "You lost #{skip_count} tweets"
end

puts "Starting to track: #{@keywords}...\nLanguages: #{@languages}"
@client.filter(:track => @keywords, :language => @languages) do |status|
  @peak_detector.collect_frequency

  @bell.ring! if @peak_detector.is_this_a_peak? || status.text.index(@magic_word)
end

https://github.com/lfcipriani/tilingol/blob/master/tilingol.rb#L40

You can also download the data captured during the game if you want to try plotting it in other ways.

Ringing the bell

Once you have the implemented goal-detection algorithm, your device should then ring the goal bell. I decided to use a Raspberry Pi for this. This was the first time I played with it, and I found its potential to be amazing. I built the bell using materials I had at home and borrowed some other items from friends. Here’s the parts list with a brief explanation on how each piece was used:

• Raspberry Pi rev 2: The computer unit that ran the Streaming API connection, detected the spikes and controlled ringing of the bell;
• Servo motor: A tiny motor that allowed precise control of position, velocity and acceleration. It was used to shake the bells;
• Jumper wires: These were used to connect the Raspberry Pi and the servo motor;
• Christmas jingle bells: An appropriately exciting sound for goal notifications;
• Plastic coffee mixer: This was used to increase the movement amplitude of the servo motor;
• Paper clips: These held the bells;
• Lego blocks: These were used to build the structure that held the motor and the Raspberry Pi… not to mention making the project look cool and nerdy.

Here’s a graphic representation of the connections, all fairly simple:

Image 5: Schematic of the hardware project. Use at your own risk!

The code that activates the pin and does the shaking is below:

import RPi.GPIO as GPIO
import time
import sys

class JingleBells:

    def __init__(self, pin):
        self.pin = pin
        GPIO.setmode(GPIO.BCM)
        GPIO.setup(self.pin, GPIO.OUT)

    def shake(self, iterations, speed):
        self.pwm = GPIO.PWM(self.pin, 100)
        self.pwm.start(19)
        time.sleep(speed)
        for n in range(iterations):
            self.pwm.ChangeDutyCycle(16)
            time.sleep(speed)
            self.pwm.ChangeDutyCycle(19)
            time.sleep(speed)
        self.pwm.stop()

    #def __del__(self):
    #    GPIO.cleanup()

if __name__ == '__main__':
    jb = JingleBells(18)
    jb.shake(int(sys.argv[1]), float(sys.argv[2]))
    GPIO.cleanup()
    exit()

https://github.com/lfcipriani/tilingol/blob/master/jinglebells.py

You just need to execute the Python file and decide how many times the bell will shake (iterations parameter) and at what speed. Then we just call this program from the goal detection algorithm we previously implemented. The only important thing is to call it asynchronously, because we don’t want to enqueue the flow of Tweets coming through the Streaming connection (see https://github.com/lfcipriani/tilingol/blob/master/jingle_bell.rb).

It’s out of scope of this article to explain what PWM is and how to use GPIO library, but know that there are many good resources about that on Raspberry Pi’s website.

Wrapping up

I just showed you how easily you can detect goals using the volume of Tweets coming from the Streaming API. After identifying these goals, you can decide the best way to notify someone that it happened… but I must say, using a Raspberry Pi is a really fun way to do it. When talking about the Twitter platform, we always say that the API is a great way to connect to the pulse of the planet and here we’ve proven that it can be tapped and interpreted in fun, interesting ways!

All the code built to make this happen is available at GitHub. Also I want to thank @luisleao for helping with the hardware part during the hackathon.

One more thing

This blog post was later turned into a tech talk where I detailed a bit more the hardware part and some challenges I faced when building this project. The slides are embedded below:

In 2014 I was asked to do a Workshop at QCon in São Paulo about Twitter Streaming APIs and I decided to build a website that would allow people to evaluate the talks just like QCon usually do for their events (green, yellow, red cards) but with tweets instead.

I built it using Twitter Streaming API and monitoring specific hashtags to increment the counter and a ruby stack: Sinatra and faye-websocket as backend, Redis as storage and jQuery and Bootstrap for the frontend.

If you interested on how to implement real time counting, how to monitor hashtags in Twitter or how to update web user interfaces using websockets you can check the source code.

Technologies used

• Twitter Streaming API and tweetstream gem
• ruby 2.x
• Sinatra (web dashboard)
• faye-websocket (Websocket middleware)
• redis (storage)
• jQuery and Bootstrap (frontend)

Installing and running

1. Clone the repo
2. Set up your Twitter credentials in config/credentials.yml with your app tokens obtained at apps.twitter.com (see config/credentials.yml.sample)
3. Set up your config/agenda.yml file with all talks, the hashtag that will trigger a vote for each one and also their attributes (see config/agenda.yml.sample)
4. Open config/initializer.rb and define:
   • Your Event hashtag (used by Twitter tracker)
   • Which words represent each level of quality (see EVENT_REVIEW_GRADES)
   • If you will be accepting only one vote per user (see ACCEPT_ONLY_UNIQUE_VOTES)
5. Install and start redis
6. Run bundle install to set up environment
7. Run foreman start
8. Access http://localhost:3000 in your browser
9. Vote! ex.: "#qconsp #twitterapi #good awesome talk"

Try to vote to a talk while having it's permalink page open to see the page updating the count

Have fun!

Implementing Sign in with Twitter is a great way to seamlessly integrate the user experience of your app or site with the Twitter platform and enable users to share content generated in your app with their followers.

The first step to implementing Sign in with Twitter is using OAuth, a widely-adopted standard that enables third parties to access resources without having to exchange passwords. Even when the specifications on how to implement OAuth are detailed and clear, it can still be a complex and frustrating task.

That’s why we built some bootstrapping code in Ruby and Python to help you get started quickly and confidently, giving you more time to focus on your product. The examples are easy to install and try. Let’s do a walkthrough in the Ruby implementation, a lightweight Sinatra app. Note that the code follows the process outlined in the Sign in with Twitter documentation.

Step 1: Obtain the request token

When the user clicks “Sign in” in your application, the first step is to obtain a request token:

Listing 1:

get '/signin' do
  # After hitting Sign in link, first thing your app must do
  # is to get a request token.
  # See https://dev.twitter.com/docs/auth/implementing-sign-twitter (Step 1)
  token = TwitterSignIn.request_token

  # With request token in hands, you will just redirect
  # the user to authenticate at Twitter
  # See https://dev.twitter.com/docs/auth/implementing-sign-twitter (Step 2)
  redirect TwitterSignIn.authenticate_url(token)
end

The request_token method is implemented in the TwitterSignIn class like so:

Listing 2:

def request_token

    # The request to get request tokens should only
    # use consumer key and consumer secret, no token
    # is necessary
    response = TwitterSignIn.request(
    :post,
    "https://api.twitter.com/oauth/request_token",
    {},
    @oauth
    )

    obj = {}
    vars = response.body.split("&").each do |v|
    obj[v.split("=").first] = v.split("=").last
    end

    # oauth_token and oauth_token_secret should
    # be stored in a database and will be used
    # to retrieve user access tokens in next requests
    db = Daybreak::DB.new DATABASE
    db.lock { db[obj["oauth_token"]] = obj }
    db.close

    return obj["oauth_token"]
end

Basically, this request will return oauth_token and oauth_token_secret and these are your temporary credentials given by Twitter to request a user access token.

Step 2: Redirect the user

Now we can redirect the user to /oauth/authenticate, passing the oauth_token obtained in the previous step.

Listing 3:

def authenticate_url(query)
    # The redirection need to be done with oauth_token
    # obtained in request_token request
    "https://api.twitter.com/oauth/authenticate?oauth_token=" + query
end

At this moment, the user will be redirected to Twitter and will be asked to authorize your app to have access to user resources. If the user grants access to your app, Twitter will redirect her or him to an URL set by you as a callback in your app configurations.

Listing 4:

# This callback will be called by user browser after
# being redirect by Twitter with successful authentication
# See https://dev.twitter.com/docs/auth/implementing-sign-twitter (end of Step 2)
get '/callback' do

  # Given that the user authorized us, we now
  # need to get its Access Token.
  # See https://dev.twitter.com/docs/auth/implementing-sign-twitter (Step 3)
  token = TwitterSignIn.access_token(params["oauth_token"], params["oauth_verifier"])

This redirection has a parameter called oauth_verifier, which is a confirmation that the user, still identified by the oauth_token obtained in the first step, has granted you access.

Step 3: Convert the request token

Now you’re going to use all the tokens obtained so far to get the access token. The access token is what’s necessary in order to — as its name suggests — access Twitter on behalf of a user. Let’s take a look at the implementation of that step:

Listing 5:

def access_token(oauth_token, oauth_verifier)

    # To request access token, you need to retrieve
    # oauth_token and oauth_token_secret stored in
    # database
    db = Daybreak::DB.new DATABASE
    if dbtoken = db[oauth_token]

    # now the oauth signature variables should be
    # your app consumer keys and secrets and also
    # token key and token secret obtained in request_token
    oauth = @oauth.dup
    oauth[:token] = oauth_token
    oauth[:token_secret] = dbtoken["oauth_token_secret"]

    # oauth_verifier got in callback must
    # to be passed as body param
    response = TwitterSignIn.request(
        :post,
        "https://api.twitter.com/oauth/access_token",
        {:oauth_verifier => oauth_verifier},
        oauth
    )

    obj = {}
    vars = response.body.split("&").each do |v|
        obj[v.split("=").first] = v.split("=").last
    end

    # now the we got the access tokens, store it safely
    # in database, you're going to use it later to
    # access Twitter API in behalf of logged user
    dbtoken["access_token"] = obj["oauth_token"]
    dbtoken["access_token_secret"] = obj["oauth_token_secret"]
    db.lock { db[oauth_token] = dbtoken }

    else
    oauth_token = nil
    end

    db.close
    return oauth_token
end

The method access_token do a request to POST /oauth/access_token passing app consumer keys and oauth tokens inside OAuth signature; and the oauth_verifier obtained in Step 2 in the body of request. With that set of information, Twitter can validate this request to the access token as legitimate and return to the app the access tokens. That in turn will enable your app to do requests on behalf of the user who just granted you access. You can now store these tokens safely in your database.

Step 4: Use the access token

This app implements a feature that lets you create a friendship between this user and an account you choose as an example of what can be done once you have been granted access.

Listing 6:

# Building oauth signature vars to use in this request
# Basically, it's our app consumer vars combined
# with user access tokens
oauth = @oauth.dup
oauth[:token] = dbtoken["access_token"]
oauth[:token_secret] = dbtoken["access_token_secret"]

# A POST request in https://dev.twitter.com/docs/api/1.1/post/friendships/create
# to make the logged user follow the ACCOUNT_TO_FOLLOW
response = TwitterSignIn.request(
    :post,
    "https://api.twitter.com/1.1/friendships/create.json",
    {:screen_name => ACCOUNT_TO_FOLLOW},
    oauth
)

Summary

Now that you know how to implement Sign in with Twitter, check the full source code to have a big picture of how the app works. If you are a Pythonist, check @jaakkosf’s Python implementation at https://github.com/jaakko-sf/twauth-web.

Have you built a Sign in with Twitter integration in another language besides English that can easily be converted to standalone code? Share it with us by Tweeting a Github link to your implementation @TwitterDev. We will update this post with implementations in other languages.

O texto que segue é de um post compartilhado na Campus Party.

Quer saber como criar um robô como esse?

O Twitter fará duas palestras na Campus Party Brasil para mostrar o potencial da Plataforma, suas APIs e o código do @oquevernaCP. A agenda é a seguinte:

• Quarta, 29/01 as 14:30 - O Twitter e suas APIs de Streaming. Palco #pitagoras
• Quinta, 30/01 as 10:30 - Workshop: Capture o momento com as APIs do Twitter. Palco #workshop2

Como tweetar para @oquevernaCP?

Se quiser apenas obter uma sugestão rápida, apenas crie um tweet com o texto "@oquevernaCP" que ele vai retornar a próxima palestra ou evento do calendário. Você também pode usar as hastags #agora ou #depois.

Tecnologias utilizadas

• ruby 2.0.0
• API de Streaming do Twitter (GET user)
• Redis
• gem tweetstream

Se estiver interessado em como o bot foi feito, você pode acessar o código-fonte.

1. knowledge of how your resources are consumed by users;
2. understanding of how HTTP caching protocol works (HTTP headers everywhere);
3. patience to solve problems when tools don’t honor the protocol (believe me, this is very common)

The first one is up to you. The third one is an inherently issue of every computer system and you should be used to that too. Time and experience will help you to deal with this burden, but there’s always mailing lists and web search to save the day.

This post is an illustrative contribution to item 2 and presents the four basic flows of HTTP caching that can be implicitly extracted from the RFC. These flows shows how clients, server and caching software will behave depending on the status of the resource being requested and its location in the topology. And by “basic” I mean that they could vary depending on the cache strategy, so use them as a starting point. Let’s go!

Cache Miss

Cache Hit

Curious fact: this animation length is half of the “Cache miss” length. 😉

Cache Revalidation (Condition False)

Cache Revalidation (Condition True)

If you want to dig deeper on the subject, this content above was included in a talk I presented at Rubyconf Brazil 2013 about HTTP Caching (intro and good practices). The slides are embedded below:

You might also like to explore the HTTP headers RFC.

The solution described in this post is simple, but uses a lot of tools and services. If you have patience, the final result is very useful. Here’s the list of things we use:

• APIfy to scrape the docs page (Scrapify is used by APIfy)
• XCPath Bookmarklet to don’t make us learn XPath again
• curl to retrieve the json version of the API docs
• jq to scrape this json and generate XML \o/
• grep to filter based on our search query
• Alfred workflows to implement autocomplete (Powerpack needed)

There are some very known tools in this list, and maybe some new to you. My recommendation is to keep attention and perceive the real value of APIfy and jq. These are really useful tools that can be used for a lot of other applications. Of course the other ones are interesting as well.

My wish was: be able to trigger a search in Alfred that shows me a list of Twitter REST API endpoints and when hitting enter, the browser would open the API endpoint doc.

The result was:

#productivity #win

Getting the data we need

APIfy is a really great tool to easily scrape an HTML page and exposes the relevant content as a json representation. The website has very descriptive tutorials that show how to create an API, I recommend them. But the process it’s uses is very simple:

For each field your API returns:

• Use XCPath Bookmarklet to get the XPATH or CSS of the element value you want;

• Do a mapping between the field name and value in APIfy editor;

You can check the brand new API created to expose all Twitter API endpoints as json at TwitterRESTAPIv1\ APIfy page. And of course, check the json representation returned by the API.

It’s very very useful. Make sure to check other APIs and how they are done, or try the Scrapify gem used to build APIfy.

Converting and filtering the data

Now that we have a json representation with each API endpoint, description and link to the docs, we can manipulate this json to be in the format needed to the next step in our flow.

jq, as website says, it is a lightweight and flexible command-line JSON processor. It’s a mandatory tool if you deal with a lot of json APIs, since it could transform raw API responses to beautified colorful representations and also allow you to filter, process, transform the data the way you want it. There’s a very good tutorial on the website.

In our case, we need to convert the json to XML… XML??? Why?!?

Well, this is the format chosen by Alfred to show autocomplete results. Basically, anytime a user trigger the workflow, a script is executed with the query and the response should be a XML with items that match the query and have other metadata such as what URL should be open if the user choose that item.

Let’s show the jq command we use:

And now let’s break it in pieces to understand what it does.

Surrounding the commands are two echo commands to add the root XML element needed to be input in Alfred. The second command is the request to the API we created that is pipelined to jq.

The jq command iterates over all entries in the response and do a string interpolation setting the resource of the endpoint and the link to it to some xml values and properties. It’s important to note that jq isn’t just to convert json to XML, it has a very powerful query language that enable you to do practically anything you want with your json. Check their tutorial.

The XML generated by jq is pipelined to a grep command, that filters for “search” using a regular expression. The final result is the following XML (already filtered by endpoints that matches the “search” keyword):

This is the input Alfred needs to show the items in autocomplete, as we saw in the beginning of this post. After testing the command in the terminal, then we need to use a Script Filter module of Alfred Workflows to run that code after triggering the keyword:

Note that instead of doing the request everytime, I just saved the json on my computer since I’m not expecting that the docs will be changed so frequently, and also because I don’t want to abuse the APIfy service.

The final version of the workflow will look like this:

That’s it! Now we have everything working:

This combination of tools could help you to build any search to your most used APIs. And of course that there are some minor improvements to the command showed above, but the beauty of this is what we could accomplish with a short period of time thanks to these awesome tools.

• Vídeo da apresentação e slides no site da QCon
• Uma entrevista que eu dei pra eles sobre o assunto

E os slides:

Abril Mídia is one of the largest media companies in Brazil. In 2011 it published 52 titles and is a leader in 22 out of 26 segments in which it operates. In the digital world, the big challenge Abril faces is deliver a large volume of information that it generates in a high diversity of digital channels, such as websites, mobile sites and TVs. The chosen solution was the building of a distributed platform, named Alexandria, that allows the content to be added and handled by separated information domains, implemented with REST constraints and with a hypermedia API, that is exposed to publishers and end users.

The result of this work became a paper published by Springer in the book "Rest: Advanced Research Topics and Practical Applications" and it was written by Luiz Rocha and me.

I spoke in QCon Conference São Paulo in 2012 about all the technical aspects of Alexandria, you can find links to slides (in Portuguese) and the talk video (in Portuguese).

Few weeks ago I gave an introduction to Semantic Web at the Sao Paulo Ruby User Group (GURU, in portuguese). Since Semantic Web is a very broader term that entail tons of technologies and tools, I decided to focus more on explaining the single main reason that make companies apply it in their projects:

Data Integration

We, as software engineers and product managers, face a lot of problems, specially in big companies, when we need to integrate data coming from several sources. Anytime I needed to do this, I realize that I kept rumbling to myself: "Why we don’t have only one universal metadata for our data? How we let this happen?"

The big triumph provided by the Semantic Web set of technologies, specifications and tools is to enable you, as owner of highly variable data and metadata, to organize it in a way that you can find any information and derive knowledge as easily as a SQL-like query. This is possible because we start to represent the data in the simplest way possible: a triple (subject -> predicate -> object). For example:

• Abril_Engineering_blog > is_owned_by > Abril
• Abril_Engineering_blog > is_hosted_by > Tumblr
• Luis_Cipriani > work_at > Abril

By establishing relationships with all your data based on a controlled and known metadata (better known as ontology), you create the most flexible format for representing a data, and from this point, you can derive anything the quality of your data would allow.

At Abril we have a perfect environment to use this strategy for data integration, since we produce a lot of content that we are able to extract structure, such as, people, events, venues, articles, user comments, etc. Some projects are on their way and others are about to start. We hope to be talking about it sooner.

Meanwhile, check the presentation slides below, it has more detail about the technologies and cases (in Portuguese):

Given the large audience of the websites at Abril, we evaluated and adopted HBase as the storage for the user timelines. We later shared our experiences in some tech events in São Paulo.

The slides are available in the link below:

We built a simple Sinatra ruby web app, with Redis as persistent storage, to track these user clicks and in 1 month we served around 1 million hits per day for 17 different websites from Abril.

Later I did a presentation about the project in a Ruby User Group Meetup in Brazil, the slides are available in Slideshare:

Being victorious through a means so amazing it cannot possibly be achieved without cheating.

- Urban Dictionary

cheatorious is a generator of simple, searchable, shareable, modular command-line cheatsheets.

Cheatsheets are very good to start learning or keep knowledge of some language or tool, but very often you just don't have the proper cheatsheet at hand when you need to remember that cryptic keyboard shortcut. Furthermore, wouldn't be nice to keep or create your own cheatsheets in a neat ruby DSL? Wouldn't be great if you have easy access to them, such as via command-line interface?

TL;DR;

• Install Cheatorious: gem install cheatorious
• Import a cheatsheet from cheatorious cheatsheet repository.
• Create your cheatsheet
• Check examples folder in repository
• Type cheatorious search CHEATSHEETNAME KEYWORD or cheatorious view to access your cheatsheets
• If you use Alfred 2, download the Cheatorious workflow in examples folder

Install

gem install cheatorious

Cheatorious is compatible with ruby 1.8.x, 1.9.x, 2.0.x, rubinius, jruby, ree and more.

Creating your Cheatsheet

Create a file and use the following syntax, shown in the example below:

cheatsheet_for "Simple VIM" do
    # put some info about you, if you want to share this later
    description "A simple VIM cheatsheet for tests"
    author      "Luis Cipriani", "github.com/lfcipriani"
    version     "1.0.0"

    # you can configure some keyboard keys variables, you can change it later to fit your personal preferences
    key :control, "^"
    key :esc    , "<esc>"
    key :leader , ","

    # your prefered keyboard keys separators (default is empty string)
    key_separator " "

    # this is an cheatsheet entry, the first parameter is what the entry does, the other are the shortcuts or descriptions
    __ "Enter insertion mode", "i"
    __ "Exit insertion mode" , _esc

    # you can create sections, that will be searchable
    section "Basic Movement" do
        __ "character left, right, line up, line down", "h", "l", "k", "j"
        __ "word/token left, right"                   , "b", "w"
    end

    section "Scrolling" do
        # this is how you use the pre-configured keyboard keys
        __ "scroll line up, down", (_control "E"), (_control "Y")
        __ "scroll page up, down", (_control "F"), (_control "B")
        __ "crazy scroll", (_leader _control "A") # this is just to show you can combine keys \o/
    end

    section "Files" do
        __ "Open file", ":e"

        # you can go infinitely deep with sections (seriously)
        section "Saving" do
            __ "Save file", ":w"
        end
    end
end

Using cheatorious

The basic CLI usage goes as follows:

$ cheatorious                                        # get help
$ cheatorious view CHEATSHEET [OPTIONS]              # view a CHEATSHEET. The CHEATSHEET variable could be a name (for imported cheatsheets) or a file that describes a cheatsheet.
$ cheatorious search CHEATSHEET [KEYWORD] [OPTIONS]  # search for KEYWORD in CHEATSHEET entries only. The CHEATSHEET variable could be a name (for imported cheatsheets) or a file that describes a che...
$ cheatorious alias NAME CHEATSHEET                  # return a shell alias command with NAME for easy access to searching a CHEATSHEET. The CHEATSHEET variable must be an imported cheatsheet. Exampl...
$ cheatorious edit CHEATSHEET                        # edit an existent CHEATSHEET. Will open it in the default terminal editor, use CHEATORIOUS_EDITOR environment variable to use another.
$ cheatorious import FILE                            # import a cheatsheet description FILE. Check https://github.com/lfcipriani/cheatorious to learn how to create your own cheatsheets. Check https:/...
$ cheatorious list                                   # lists the available cheatsheets. See 'import' command.
$ cheatorious reload CHEATSHEET                      # reload a CHEATSHEET after editing it with 'edit' command
$ cheatorious remove CHEATSHEET                      # remove a CHEATSHEET. The original file is kept for later recovering
$ cheatorious writers [OPTIONS]                      # lists the available writers or set a default
$ cheatorious help [COMMAND]                         # Describe available commands or one specific command

Searching

To search on your cheatsheets:

$ cheatorious help search
Usage:
  cheatorious search CHEATSHEET [KEYWORD] [OPTIONS]

Options:
  -s, [--section]        # matches KEYWORD only on section names, returning all entries and sections inside it.
  -r, [--reverse]        # reverse means to search only the values of a cheatsheet (and not entries, as usual). For example, search by shortcuts.
  -S, [--sensitive]      # case sensitive search (insensitive is default).
  -w, [--writer=WRITER]  # writer to use for the output. If not set, uses the default.

search for KEYWORD in CHEATSHEET entries only.
The CHEATSHEET variable could be a name (for imported cheatsheets) or a file that describes a cheatsheet.
Omit KEYWORD to view the full cheatsheet.

Alias

If you are tired to type everything above to do a simple search, use the alias command:

$ bin/cheatorious help alias
Usage:
  cheatorious alias NAME CHEATSHEET

return a shell alias command with NAME for easy access to searching a CHEATSHEET.
The CHEATSHEET variable must be an imported cheatsheet.
Example: cheatorious alias svim simple_vim >> ~/.bashrc
         next time just use: svim KEYWORD [OPTIONS]

It will just show the alias command, it's up to you to decide where to put it.

Editing, reloading, removing

$ cheatorious edit CHEATSHEET    # edit an existent CHEATSHEET. Will open it in the default terminal editor, use CHEATORIOUS_EDITOR environment variable to use another.
$ cheatorious reload CHEATSHEET  # reload a CHEATSHEET after editing it with 'edit' command
$ cheatorious remove CHEATSHEET  # remove a CHEATSHEET. The original file is kept for later recovering

Writers

Cheatorious has a default Text writer to give a fairly nice output for your cheatsheet. Since we can have multiple writers avaiable, you can list and set a default writer through the command writers:

$ bin/cheatorious help writers
Usage:
  cheatorious writers [OPTIONS]

Options:
  -d, [--default=DEFAULT]  # set a default writer for next searches.

lists the available writers or set a default

I'll work to provide more options of output as soon as possible (markdown, colored, HTML, etc), but you can also contribute writing your own writers and sending a pull request to this project. Check the examples folder to see a Writer sample, you just need to implement that interface.

Tips to improve the experience!

• Create aliases to save typings in the cheatsheets you use more
• Sync your cheatsheets with Dropbox by creating a symlink for the ~/.cheatorious folder

The slides below were done basically after reading Caching RFC, so if you want a visual reference of web caching, here it goes:

Later in 2013 I presented another talk at Rubyconf Brazil giving more context about caching strategies, you can find the article in this website as well.