The things you may discover when going out for a walk

Published on

I quit my job at Haiku Learning a few weeks ago. It was an awesome job. I worked in very interesting projects that gave me the opportunity to learn; from designing graphical charts with D3JS, to creating API servers and clients, to micro-services and Rails Engines. Thanks to all these projects I have a very solid command of Ruby on Rails and the design principles that guide the creation of maintainable code.

But I lost something when I started working full time. I lost the opportunity to create my own products. My own projects. And that’s exactly where my passion lies.

So quit my job.

I joined a MBA in the university Pablo De Olavide to improve my business skills. I will tell you how that goes :)


Writing an RSpec test for an asynchronous action

Published on

In the previous article you learnt how to write a Rails action that can handle concurrent users, an asynchronous action. Today you are going to learn how to write an RSpec test for it.

This is the controller we want to write a test for:

class ApplicationController < ActionController::Base

  def sample
    EM.defer do
      render json: { response: 'Hello World' }

      request.env['async.callback'].call response
    end

    throw :async
  end

end

If this were not an asynchronous action, we would simply write a test like this:

require 'spec_helper'

describe ApplicationController, :type => :controller do
  it "hello world" do
    get :sample
    expect(JSON.parse(response.body)['response']).to eq('Hello World')
  end
end

But this won’t work because throw :async will kill the action right away. throw :async will make the interpreter go to this line of code defined in the Thin gem. The problem is that Thin is not loaded at all when running a controller test.

In order to properly test an asynchronous action you need to run the test using a real Thin server. You can use Capybara to do that.

Gemfile

gem 'thin'
gem 'capybara'
gem 'selenium-webdriver'

In spec/rails_helper.rb:

require 'capybara/rails'

Capybara.default_driver = :selenium

Capybara.server do |app, port|
  require 'rack/handler/thin'
  Rack::Handler::Thin.run(app, :Port => port)
end

Write the test:

describe ApplicationController, :type => :feature do
  it "my test" do
    visit some_path
    expect(page).to have_content('Hello World')
  end
end

And that’s it. Happy testing.


How to handle concurrent requests in rails

Published on

Let’s say we have this action:

class ApplicationController < ActionController::Base

  def normal
    sleep 5 # Some code that takes a long time. For example: a web request.
    render plain: 'Hello'
  end

end

This code is a scalability problem, because each request efectively block other requests from being handled. If your service receives more than one request per 5 seconds, your service will start failing to respond. But it doesn’t need to be like that. There is a way to tell Rails that it is okay to handle some other request while some other work is being done and then go back to that original request when all it’s ready.

It’s called concurrent mode.

In this example we will use Thin plus rails’s concurrent mode to handle request concurrently.

The first step is to add the Thin server to the Gemfile. A simple gem 'thin' in the Gemfile will do.

The second step is to activate the concurrent mode in the rails environment. For example, in development:

# config/environments/development.rb
Rails.application.configure do

  ...some other settings

  config.allow_concurrency = true
end

Then set up the action that you want to handle asyncronously in this way:

class ApplicationController < ActionController::Base
  def async
    EM.defer do
      sleep 5 # Some work that take a long time.
      request.env['async.callback'].call response
    end

    throw :async
  end
end

Start the server like this: bundle exec thin --threaded -p 5500 --threadpool-size 50 start

That means that Thin will be started in threaded mode, and it will use up to 50 threads. The threaded option is important as Thin defaults to non threaded mode.

When the request is being handled, the EM.defer block will be executed in a separate thread. At the same time the throw :async will be executed, which basically tells the Thin server that this request will be handled asyncronously and that it should inmediately start working on a different request. request.env['async.callback'].call response will communicate the response to the Thin server and send it back to the client.

Let me remark that, until request.env['async.callback'].call response is executed, no response is sent back to the client.

There is one important gotcha with throw :async: No rack middleware will be executed for the response. Thereby if you try to set up a cookie it will fail. But here is a way to fix that:

class ApplicationController < ActionController::Base
  def async_with_cookies
    EM.defer do
      sleep 5
      cookies[:message] = 'hello'
      cookies.write(headers)
      request.env['async.callback'].call response
    end

    throw :async
  end
end

cookies.write(headers) will take care of setting the headers that needs to be set in order to make the changes in the cookies. It’s basically the code that gets executed when the Cookie middleware is executed.

And that’s it, so simple.

One way to test that you action can really handle concurrent requests to to query the url with multiple requests at the same time. You can do it with Apache HTTP server benchmarking tool, packaged with Mac OS.

time ab -c10 -n10 http://127.0.0.1:5500/async_with_cookies
# That will send 10 requests, 10 request at a time (so all go in at the same)

➜  Concurrent  time ab -c10 -n10 http://127.0.0.1:5500/async_with_cookies
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient).....done


Server Software:        thin
Server Hostname:        127.0.0.1
Server Port:            5500

Document Path:          /async_with_cookies
Document Length:        2844 bytes

Concurrency Level:      10
Time taken for tests:   5.178 seconds
Complete requests:      10
Failed requests:        7
   (Connect: 0, Receive: 0, Length: 7, Exceptions: 0)
Write errors:           0
Non-2xx responses:      5
Total transferred:      34886 bytes
HTML transferred:       32330 bytes
Requests per second:    1.93 [#/sec] (mean)
Time per request:       5178.076 [ms] (mean)
Time per request:       517.808 [ms] (mean, across all concurrent requests)
Transfer rate:          6.58 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:   196 2687 2614.2   5158    5178
Waiting:      196 2687 2614.2   5158    5177
Total:        196 2687 2614.2   5159    5178

Percentage of the requests served within a certain time (ms)
  50%   5159
  66%   5159
  75%   5164
  80%   5176
  90%   5178
  95%   5178
  98%   5178
  99%   5178
 100%   5178 (longest request)
ab -c10 -n10 http://127.0.0.1:5500/async_with_cookies  0,01s user 0,02s system 0% cpu 5,207 total

So you can see that it took 5 seconds total to handle the 10 requests (each one taking 5 seconds) thereby proving that we can handle concurrents requests. This is a very useful model for handling web requests where one portion of the request may take a few seconds. For example at Haiku we use this technique when handling concerning Google signin. The portion of code where we need to wait we simply executed them in a Event Machine defered block.

Note: If you use Pow for your app, use the ip address as in the example, because Pow will only handle one request at a time.

You can see a full example here: https://github.com/Nerian/concurrency


The 16 most commonly asked questions about Git

Published on

How to edit an incorrect commit message in Git

If the commit is the last one:

git commit --amend -m "New commit message"

You can also use rebase to edit the commit messages – choose ‘reword’ for the commits you wish to edit.

git rebase -i HEAD~5  # Rebase the last 5 commits

Warning: Doing a rebase does change the commit history – it changes the SHA of commits – so do not rebase a commit that has already been published otherwise you will end up with duplicated commits when your team merges your branch with their own branch.

Protip: You can ignore that rule if you are working on a feature branch and you are about to merge your work with master and you just want to clean up things. Since you are going to be merging to master and then deleting the feature branch nobody is going to be affected. It is actually a good practice to do that in such a case.

Trivia: You can’t rebase the first commit of a repository, unless you pass the --root option.

How to undo the last Git commit

git reset --soft 'HEAD^'

Protip:
git config --global alias.undo-commit 'reset --soft HEAD^'
git undo-commit

How to delete a git branch both locally and remotely:

git push origin --delete <branchName>

What is the difference between ‘git pull’ and ‘git fetch’

  • git fetch: Updates your local repository with the data from remote.
  • git pull: Updates your working copy with the changes in the remote.

In practice: If you want to get the changes from remote without immediately changing your working copy then use git fetch. Otherwise use git pull. git pull does a git fetch followed by a `git merge.

How to undo ‘git add’ before having commited

git reset <file>
git reset # omit the file in order to reset all of them

Protip:
git config --global alias.unstage 'reset HEAD --'
git unstage <file>

How to merge a Git conflict

A conflict in Git occurs when two branches happen to modify the same area of code. Say you have a branch A:

# John renamed this method to:
def john_is_the_best
end

…while branch B have this code in the same line:

# Jaime renamed this method to:
def jaime_is_the_best
end

In such case Git is not capable of merging both changes because Git can’t possibly know which one to choose. Your manual intervention is necessary and that is what we call a Git conflict. Git merging tools will markup areas in your code with conflict markers.

After seeing a conflict, you can do two things:

  • Decide not to merge. git merge --abort can be used for this.

  • Resolve the conflicts. Git will mark the conflicts in the working tree. Edit the files into shape and git add them to the index. Use git commit to seal the deal.

Protip:
git config merge.conflictstyle diff3 # Uses diff3 conflict markers.

Anatomy of a conflict marker:

<<<<<<<
Changes made on the branch that is being merged into. In most cases,
this is the branch that I have currently checked out (i.e. HEAD).
|||||||
The common ancestor version. This is what the common ancestor looked like. This is useful because you can compare it to the top and bottom versions to get a better sense of what was changed on each branch, which gives you a better idea for what the purpose of each change was.
=======
Changes made on the branch that is being merged in. This is often a
feature/topic branch.
>>>>>>>

Following the example, if we try to merge John’s branch in Jaime’s branch we would have had:

<<<<<<< HEAD
def jaime_is_the_best
||||||| merged common ancestors
def who_is_the_best?
=======
def john_is_the_best
>>>>>>> john

Understanding the intention behind each diff block is generally very helpful for understanding where a conflict came from and how to handle it.

This shows all of the commits that touched that file, considering just changes in between the common ancestor and the two heads you are merging, so it doesn’t include commits that already exist in both branches before merging. This helps you ignore diff blocks that clearly are not a factor in your current conflict.

git log --merge -p <name of file>

After resolving the conflicts, it is a very good practice to test that you didn’t broke anything. Run your automated test suite.

The easiest conflicts to solve are the ones that never happened:

  • Talk to your team and if you anticipate that some of you are going to be doing extensive modifications to a single file consider instead working sequentially – one of you make your changes first, and the other can work on the top of said changes. No conflicts.
  • If working in parallel is a must, then merge assiduously; that way you will catch conflicts sooner than later – and so will be smaller and fresher in the memory of both parties involved.

If there is something worse than a difficult merge is a merge that went wrong but got commited anyways. If you find yourself unable to resolve the conflicts with confidence, do not merge, instead abort the merge with git merge abort and talk to your team on how to best tackle it.

How to clone all remote branches with Git

git remote update
git pull --all

How to remove all untracked files in Git

git clean -f # Remove all untracked files.

git clean -f -d # Remove all untracked files and directories.
git clean -f -X # Remove just ignored files.
git clean -f -x # Remove all untracked and ignored files.

Protip:
Use the --dry-run option in order to preview changes.

How to remove a git submodule

git submodule deinit <name>

How to make an existing Git branch track a remote branch

git branch -u upstream/foo foo # you can omit the last 'foo' if you are already in that branch.

How to do a “git export” (like “svn export”)

git archive --format zip --output name.zip master

How to rename a local Git branch

git branch -m <oldname> <newname> # You can skip oldname if you want to rename the current branch.

How to add an empty directory to a git repository

Currently the design of the git index (staging area) only permits files to be listed. Directories are added automatically when adding files inside them. That is, directories never have to be added to the repository, and are not tracked on their own.

If you really need a directory to exist in checkouts you should create a file in it. For example a .gitignore file; you can leave it empty.

How to checkout a remote branch

Checkouts a branch named my_branch that exists in any of the remotes.

git checkout my_branch

If multiple remotes have branches with the same name you will get an error – in which case you need to use the next form.

Protip:
Use the next form to checkout a branch named my_branch that exists in the remote named 'origin'

git checkout origin/my_branch

How to revert to previous Git commit

There are two cases:

  • If you have published it already:
git revert commit_sha
git revert commit_sha1 commit_sha2 commit_sha3 # Three commits are reverted in 3 separate commits
git revert HEAD~2..HEAD # Revert a range of commits. Each commit will have is own revert commit.
  • If you haven’t publish it already
git reset commit_sha # The one you want to revert to.

Protip:
If you have staged changed and you don't want to keep them use the `--hard` option.
git reset --hard commit_sha

How to force git to overwrite local files on pull

Update from remote.

git fetch --all

Reset the master branch to what you just fetched. The --hard option changes all the files in your working tree to match the files in origin/master, so if you have any local changes, they will be lost. With or without --hard, any local commits that haven’t been pushed will be lost.

git reset --hard origin/master

How to stash only one file out of multiple files that have changed

This will stash everything that you haven’t previously added. Just git add the things you want to keep, then run it.

git stash --keep-index

Total Terminal

Published on

As a developer you probably access your terminal hundreds of times a day. If you read the first article on how to be a productive developer you already have each application on a space; and one space per application. So in order to get to your Terminal you need to switch spaces. One space.

But there is a better way. What if you could have your Terminal in the space that you are right now, but only visible when you need it? Eliminating context switching is the key to regain your focus and your ability to solve problems real quick. You can have that with one single tool: Total Terminal.

It starts when you computer starts but it remains invisible until you need it. When you need it, you need only to press a shortcut key and a terminal appears at the top of your screen. One keystroke and it’s gone.

Start saving time now: http://totalterminal.binaryage.com/

This article is part of a series on how to create the best development environment. Subscribe so you don’t miss any chapter!


Older ›