Include vs. Extend…why not both?

If you don’t already know, there is a subtle difference between including and extending a module into one of your classes. include allows you to access instance methods while extend allows you to access class methods. See the example below for clarification:

module FooModule
  def foo
    puts 'foo'
  end
end

----------

class Bar
  include FooModule
end

Bar.new.foo => 'foo'
Bar.foo => NoMethodError: undefined method 'foo'

----------

class Bar
  extend FooModule
end

Bar.new.foo => NoMethodError: undefined method 'foo'
Bar.foo => 'foo'

So what if you want to create a module that will grant you access to both class and instance methods simultaneously? Luckily, using Rails’ ActiveSupport::Concern will help you separate your instance and class methods easily.

module FooModule
  extend ActiveSupport::Concern

  def foo
    puts 'foo'
  end

  module ClassMethods
    def baz
      puts 'baz'
    end
  end
end

----------

class Bar
  include FooModule
end

Bar.new.foo => 'foo'
Bar.foo => NoMethodError: undefined method 'foo'
Bar.new.baz => NoMethodError: undefined method 'foo'
Bar.baz => 'baz'
Advertisements

How to change test environment variables

There are times when you might want to change your environment variables within a testing environment. Maybe you want to communicate with a different server or want to enable some kind of functionality that is normally disabled for testing. You can use stubbing to elegantly and temporarily just set your environment variables within a test.

ENV.stub(:[]).with('FEATURE_ENABLED').and_return('true')

Here, I am enabling a feature that allows us to access user information. Normally, the lookup is computationally expensive so we don’t run it on all of our tests. But when the need arises, it’s good to be able to change your environment variables easily within a single test.

The limits of pg_search

At work, I’ve been tinkering with Postgres and trying to optimize the search that we have implemented for our company’s app. For those of you who don’t know, Postgres comes with a lot of great built in functionality to carry out full text search on your database. In our Rails app, we use a gem called pg_search that makes use of a lot of these features to quickly and easily set up a search engine for your website. Let’s say you wanted to be able to search through a list of contacts; with pg_search, you just have to set up a simple search scope.

class Contact < ActiveRecord::Base
  pg_search_scope :search, against: [:first_name, :last_name]
end

Now we can simply call the search method on the contacts table to find records quickly and easily.

Contact.search('Tim')

We make use of pg_search’s other search function, called multisearchable. The code required to set it up is very similar to a simple pg_search_scope.

class Contact < ActiveRecord::Base
  multisearchable against: [:first_name, :last_name]
end

The difference is that multisearchable utilizes a PgSearch::Document, which is a record that points back to the original search object. So if you wanted to search across multiple tables, PgSearch::Documents allow you to query the pg_search_documents table and pull up any records belonging to a specific type. The great thing about multisearchable is that it returns ActiveRecord::Relations, which means that you can chain commands onto your search result.

PgSearch.multisearch('Tim').where(searchable_type: 'Contact')
PgSearch.multisearch('Tim').where(searchable_type: 'Contact').where(last_name: 'Park').limit(5)

You can even make use of more advanced search techniques such as employing use of dictionaries, indexes, and weighting.

PgSearch.multisearch_options = {
  using: {
    tsearch: {
      prefix: true,
      dictionary: 'english'
    }
  }
}

One of the problems we come across is that while multisearchable is convenient and fast, it takes up a lot of space. Any time you create a new record, you are also creating a PgSearch::Document to point to it. And because our PgSearch::Documents make use of a GIN-indexed tsvector column, creating and updating our pg_search_documents table is taxing on our system and an expensive use of space. Unfortunately, we can’t use the simple pg_search_scope because it doesn’t really support searches that perform table JOINs.

In an effort to try and remove the need for the pg_search_documents table, I tried to create a method that would be able to search our database without the need for pg_search and multisearchable. I began by indexing all of the tables we search with a tsvector, which simply represents documents into an easily searchable form. Thoughtbot published a great blog post detailing how to generate a migration to create an indexed tsvector on your tables. We use GIN indexing, but it’s likely that for smaller apps, a GIST index will be better. For more information, you can read the Postgres documentation about it, but here’s the main takeaways:

In choosing which index type to use, GiST or GIN, consider these performance differences:

  GIN index lookups are about three times faster than GiST

  GIN indexes take about three times longer to build than GiST

  GIN indexes are moderately slower to update than GiST indexes, but about 10 times slower if fast-update support was disabled (see Section 54.3.1 for details)

  GIN indexes are two-to-three times larger than GiST indexes

As a rule of thumb, GIN indexes are best for static data because lookups are faster. For dynamic data, GiST indexes are faster to update. Specifically, GiST indexes are very good for dynamic data and fast if the number of unique words (lexemes) is under 100,000, while GIN indexes will handle 100,000+ lexemes better but are slower to update.

Ultimately, I was able to create an ‘alt_search’ method that executed a raw SQL query to return a list of results. It’s important to note that on the contacts table, the :tsv column is a tsvector of :first_name, and :last_name and the :tsv column on the addresses table is a tsvector of :address, :city, :state and :zip_code. The problem with the SQL query is that it return a hash where each ActiveRecord::Relation is returned as a JSON object. To end up with an array of ActiveRecord::Relations, I had to do an ActiveRecord query through the contacts table once again to find our results by ID.

def self.alt_search(query)
  sql = <<-SQL_QUERY
    SELECT contacts.id
    FROM contacts
    INNER JOIN (
      SELECT contacts.id AS pg_search_id,
      (
        ts_rank(
          (contacts.tsv),
          (to_tsquery('english', '#{query}')), 0
        )
      ) AS rank
      FROM contacts
      WHERE (
        ((contacts.tsv) @@ (to_tsquery('english', '#{query}')))
      )
      ORDER BY rank DESC
      OFFSET 0
    ) pg_search ON contacts.id = pg_search.pg_search_id

    UNION

    SELECT contacts.id
    FROM contacts
    INNER JOIN (
    SELECT addresses.id AS pg_search_id,
      (
        ts_rank(
          (to_tsvector(addresses.city)),
          (to_tsquery('english', '#{query}')), 0
      )
    ) AS rank
    FROM addresses
    WHERE (
      ((addresses.tsv) @@ (to_tsquery('english', '#{query}')))
    )
    ORDER BY rank DESC
    OFFSET 0
  ) pg_search_phones ON contacts.phone_id = pg_search_phones.pg_search_id
  ;
SQL_QUERY
  @matched_contacts = []
  results = ActiveRecord::Base.connection.exec_query(sql)
  if results.present?
    results.each do |msg_hash|
      @matched_contacts << (msg_hash['id'].to_i)
    end
    Contact.where(id: @matched_contacts)
  else
    nil
  end
end

At first, this method proved to seem promising as it was almost an order of magnitude faster than multisearchable queries. However, when you start working with large tables (millions of records), pg_search shines. The power of the PgSearch::Document is that it serves as an indexed table of its own. You can query all of your tables at once and then filter through the documents. In addition, it’s obvious that implementing pg_search is much easier. Imagine trying to refactor that SQL query when your tables change or you add new model attributes!


For now, it seems as though we’ll have to live PgSearch::Documents and look for other options. Many of the most visited websites in the world make use of Apache Solr and it seems as though we may be headed that way. But if you’re in a similar situation, I recommend that you take the time to at least GIN-index your pg_search_documents table. We have found that this increases search speeds by up to 50%, which can be dramatic when you’re searching a table with millions of records.

EDIT: After some research and experimentation, we decided to use Elasticsearch. Our Elasticsearch node is hosted by Bonsai, which is an easily available add-on through the Heroku marketplace. Elasticsearch allowed us to drop our PgSearch::Documents table, which was almost 10 GB!!! I highly recommend you look into Elasticsearch if you would like to include some search capability into your app. If you’re looking for some direction, shoot me an email!

DaVinci Coders – Bootcamp Review

It’s been over a month since I’ve written my last post. And quite frankly, I regret it. The truth is that my last month has been extremely busy. I worked on a couple of projects at DaVinci Coders, had some scheduling issues at my job, and worked through some personal difficulties. But those aren’t good excuses.

Recently, I’ve been reflecting a lot on the last month where I’ve worked on two projects for DaVinci Coders. The first, Shift Scheduler, is an app that allows people to conveniently manage employees schedules. The second, Tracker Request, uses the Pivotal Tracker and GitHub APIs to create a one-stop-shop for managing projects. I want to recap the important bits from my last month at DaVinci Coders.


Test Driven Development is an invisible savior

If you’re new to TDD, I know it can be extremely overwhelming. I used Rspec and Capybara heavily at DaVinci Coders and honestly it felt like I had to learn two new languages in addition to Ruby. But after struggling with it for a long time, I started to realize the invaluable benefit of testing.

While working on Shift Scheduler, I found that having tests enabled me to work confidently because I knew my test would help me find major bugs in my code. In addition, when a problem did occur, Rspec output would point me in the right direction instead of having to trace through the MVC architecture and figure out where the problems were occurring.

One tool that I started to use often was the byebug gem. It is an amazing debugging tool that allows you to test your code at points of failure within a rails console-like terminal.


There is a gem for everything

Go to Ruby Toolbox and search for anything your heart desires. It’s amazing to see how much effort people in the Ruby community have put into creating gems to cater to your needs. I remember a distinct moment when I was pair programming with Danny and we needed to figure out a way to format some of our json output. We spent about 30 minutes reading about the topic and then found a simple gem called Active Model Serializers that allows you to easily format json in Rails.


There is no alternative to a good team

Early in our Ruby on Rails course, we learned about pair programming and it’s role in productive coding. I tried to pair with as many of my classmates as possible and I learned that there is an art to being able to work with all manners of people. Everyone works, thinks, types, and talks at a different pace and in unique ways. And I learned so much from my classmates in seeing code through their viewpoint.

Especially while we were all working with our projects, it was so obvious how productive our teams were when everyone was communicating well and being open. Progress always stalled when people were prone to ignoring Slack messages and when individual programming was favored over pair programming. To be fair, we all had our own busy lives to deal with, but I never regretted taking time out of my day to schedule a time to pair. It’s just easier to stay focused and motivated with a partner in crime.


You can do anything…

On day one of class, I was overwhelmed with our syllabus. It covered so many topics, each of which seemed to merit a class of it’s own. And if you had told me that I had to built a fully functional web app in 13 weeks, I would have bailed out of there. But it’s inspiring to see how far I, and my fellow peers, have come in 13 weeks. Some of them came in with no coding experience and helped build amazing sites like Speed Op and Footloose. It just goes to show that you can accomplish anything you want as long as you’re willing to put in LOTS and LOTS of work. Also…


…especially if you have an amazing instructor

…it helps to have an amazing instructor, like Jason Noble. He’s a top Rails contributor, his depth of knowledge seems limitless, and most importantly – he’s extremely patient. Anytime I had a question or wanted to delve deeper into a topic, Jason had an answer or could point me in the right direction. I enjoyed his teaching style as he would often hold our hands in class for difficult concepts and then let us explore other topics on our own.

If you are looking to learn more about Ruby On Rails and want some instruction, I highly recommend looking into DaVinci Coders. If you do decide to take this course, I have a few pieces of advice for you below.


How to Bootcamp

Make friends with everyone

You can learn something from everyone. My greatest ‘AHA!’ moments came while pair programming with others. Often, I would explain a concept they were struggling with and realize that I actually knew what I was talking about. Most importantly, you can always share knowledge. There is too much information covered in this course for you to become an expert at everything. Maybe you are really good at testing, but your friend is a Git master, and the quiet classmate in the corner is secretly a JavaScript guru.

Focus on one topic at a time

In the first month of class, we covered Git, the Command Line, Pair Programming, Agile Development, Ruby, and TDD. There is ABSOLUTELY no way to learn all of this at once, so pick a topic that interests you and read about it obsessively for 3-4 days. When you feel comfortable, move onto the next topic. I found that tackling concepts in a slow piecemeal way was much less overwhelming than trying to read about all topics at once.

My advice is to get really comfortable with Git first. It’s the foundation of modern version control and allows you to fix code mistakes easily. I saw some of my classmates struggle with Git and it reduced their productivity.

Ask for help

This was my greatest weakness in class. Too often, I read at length about topics like TDD and Active Record without asking the right questions to my instructor, TAs, and mentors. The great thing about DaVinci Coders is the plethora of resources available to you. But if you get too deep in the documentation or blog posts, it is easy to forget that sometimes the right question will save you hours of aimless reading.


Overall, I’m so glad that I took the Ruby On Rails course at DaVinci Coders. I learned a lot in the last 13 weeks and even though it was stressful to complete our final projects, I had so much fun doing it. I feel confident now that I was meant to become a developer and I’m excited for this new chapter in my life. The next couple of months, I’ll be searching for work and while that will be another challenge, I can’t think of anything else that would make me happier.

Project Inception – Taking good first steps

This week in class, we started our capstone projects. It’s hard to believe that it will all be over in four weeks, but I’m grateful for what I’ve learned and it’s been an incredible journey so far. And I can’t wait for the topics we’ll cover in the coming weeks to make our projects even better. So far, we’ve covered the fundamentals of building a Rails app including using Git, coding in Ruby, Agile development, employing TDD techniques, the plethora of Ruby gems, and of course, Rails.

I’ve signed up to be a part of two projects that I think are interesting and will challenge me to be a better Rails developer. On one project team, our goal is to develop a scheduling application that will allow managers and employees to interact easily over scheduling shifts. It will contain some very useful features such as being able to request time off, switch shifts, set default availability, and set overtime.

On the other team, we are building an app that will integrate GitHub and Pivotal Tracker into one system. The goal is to be able to modify Tracker stories and merge Pull Requests all from one dashboard. We will be using the GitHub and Pivotal tracker APIs to accomplish this and I’m excited to see how this project develops over the next few weeks.


Inception – starting a new project

Once we formed our project teams, we used Agile development techniques to take the first steps toward building our apps. Before discussing any aspect of the code, we framed the big picture of our project using a whiteboard discussion. By having an open discussion with managers, developers, and clients, it is easier to envision to ultimate design and use of an application and turns a daunting task into smaller, more manageable stories. As is custom in Agile development, stories can be generated during weekly meetings and developers can work on turning those stories into code during a sprint, or iteration.

So we started our projects with a discussion of the vision we had for each application, our goals and anti-goals, and who we thought might use the software. In this way, we started to have a clear understanding of how to start building our app and it no longer seemed like a daunting task. We also broke down the design into personas, roles, and activities. Personas are people who would use the app, roles are the positions they might fill in using the app, and activities are the functions they want to be able to use.

The foundation of our development plan lies in ‘user stories.’ A user story is a brief description of how someone might use the app and reveals the functionalities that need to be incorporated into it. For example, a user story for a scheduling app might read like, “As an employee, I want to be able to request time off, so my manager can find a replacement for my shift.” I’ll go over each aspect of a project development plan and how it plays an important role in keeping a team focused and motivated.


Goals, Risks, and Anti-Goals

Goals

Create a list of goals you want to accomplish while developing your new product. This seems obvious, but you can use this opportunity to talk goals for your team instead of just features you want to enable in the software. For example, what deadlines do you want to meet and how can you align yourselves to the values of the client or your own company?

Anti-Goals

This list is equally important to the goals you set. It outlines things you might tackle in the future, but should NOT be a priority for the time being because it could detract from your team’s progress. Perhaps you don’t want to worry about compatibility on mobile devices so you can steamroll ahead toward an MVP.

Risks

Outlining potential obstacles and pitfalls can help your team focus on problems before they arise. Perhaps your software will need to use new (buggy) technologies to be implemented properly.


Personas, Roles, Activities

Personas

Personas are a list of people who will be using your software. In my shift scheduling project, our list of personas would look like visitors, users, administrators.

Roles

Once you define the list of personas, you can dive into the roles they might fall into when they start using the software. For the shift scheduler, users might fall out into various categories such as employees, managers, and payroll issuers. Each role will have certain capabilities based on their role; for example, payroll issuers should be able to see how many hours an employee worked, but not be able to modify their schedule.

Activities

At this point in the process, you should start listing out the specific details of how a user might interact with the software. This is important in starting to develop user stories. Activities could be as simple as ‘log in to the home page’ or as complex as ‘generate a list of employees scheduled to work on a certain day.’


User stories

Writing good user stories is an art in and of itself. The goal is to create a scenario that is detailed enough that it can be broken down into a development task with acceptance criteria and relevant tests. The first step is to look at your list of activities and roles to generate a very broad user story using this template:

As a <user-type>, I want <functionality>, for <benefit>.
As a visitor, I want to create a new profile, so that I can start using the application.

Once you have a basic user story, the goal is to generate a story for whatever project management tool your team uses (e.g. Pivotal Tracker). Since A tracker story needs to be tangible in a way that any software developer can start working on it, it’s important to start developing a set of acceptance criteria to test if the story is complete. In our class, we write a list of steps that a user might take to accomplish the user story.

Acceptance Criteria:

Visit the home page

Click on 'Sign Up' link

Fill in my first name
Fill in my last name
Fill in my email address
Fill in my phone number
Fill in my birthday

Click on 'Complete Sign Up' Link

Redirect to 'My Profile' with a banner that says,
'Thank you for signing up!'

Those steps eventually become a capybara test that we can implement in Rails.

require 'rails_helper'
feature 'New user sign up' do
  scenario 'should let a user create a new profile' do
    visit '/'
    
    click_link 'Sign Up'

    fill_in 'First Name', :with => 'Bob'
    fill_in 'Last Name', :with => 'Smith'
    fill_in 'Email', :with => 'bobsmith@bobsmith.com'
    fill_in 'Phone Number', :with => '1234567890'
    fill_in 'Birthday', :with => '01/01/01'

    click_link 'Complete Sign Up'

    expect(page).to have_content('Thank you for signing up!')
  end
end

Once you’ve created one user story, repeat the process until you have enough work that can be digested during one sprint. Don’t forget to keep your project on track by reflecting on your goals, risks, and anti-goals often. It’s important to have a clear, open discussion at project inception to ensure your team’s success!

Sources

Pivotal Labs – Project Inception
RallyDev – Writing great user stories
Roman Pichler – 10 tips for writing good user stories

RubyMine – Creating live templates

Time is of the essence when you’re coding. Recently, I wrote about a line of code to get rid of a huge stack trace in rspec-rails. This will help you stay sane while you try to troubleshoot failing tests in your rails app.

config.backtrace_exclusion_patterns << /\.rvm\/gems/

However, you don’t want to have to keep typing this out every time you start a new rails app. If you use RubyMine, you’re in luck because you can just save this to a live template. Live templates act as auto-fill shortcuts in the same way that RubyMine predicts when you’re trying to type ApplicationController.


Creating a live template

If you want to create a live template, just highlight a line of text that you don’t ever want to have to type again, like the one above. Then go to the Tools Menu and select Save as Live Template. It will bring up a window where you can assign that code to an Abbreviation.

Live_Templates

Then just hit okay and go into any file in RubyMine. Start typing your abbreviation and you should see a window popup with a suggestion. In the example above, I just type in back and hit TAB to let RubyMine take care of the rest.

Git – Rebasing to a specific commit

I was working on homework that challenged us to create tests to build a rails app from the ground up. I finished the homework on a master branch and decided I wanted to do it again for practice. So I created a topic branch off of an early commit (e3f8704) to do the work from scratch.

$ git checkout -b homework e3f8704

Later I realized I made a mistake and wanted to use interactive rebase to fix a commit on the topic branch. To do so, I just passed in the commit I used to branch as an option. This tells Git that you want to rebase from that commit.

$ git checkout homework
$ git rebase -i e3f8704

Resource: https://git-scm.com/docs/git-rebase