Testing RSpec model validations

I recently came across an issue while trying to write RSpec tests for model validations that I think are worth addressing. I think it’s worth testing validations because they may change over time and you want to make sure your test suite is accounting for them. For example, I have character limits for messages, which is a critical model validation. If I were to accidentally delete the validation or change the character limit, there would be a lot of unexpected consequences.

The first validation I wanted to test was a character limit on messages that was only triggered if the message was sent by a user. To do so, I created a from_user method that checks to see if the message.sender_type == 'user'. The model looks like this:

class Message < ActiveRecord::Base
  validates_presence_of :body, :status, :contact
  validates :body, length: { in: 1..420 }, if: :from_user?, on: :create

  def from_user?
    self.sender_type == 'user'
  end
end

At first, it seemed like I could easily use thoughtbot’s shoulda gem to write a simple test.

describe 'character limit for user-sent messages' do
  it {should_validate_length_of(:body).is_at_least(1).is_at_most(420).on(:create) }
end

However, the validation kept failing. After a little bit of digging, I realized it was because I never specified a subject. In particular, I never specified that my subject was a user-sent message. I found that there are two solutions to this problem.

The first is to specify a subject with a FactoryGirl.build. It’s important that you use ‘build’ and not ‘create’, because the ‘create’ method will trigger all model validations, which is exactly what you’re trying to test. In addition, when you ‘build’ the model, make sure you leave out the attribute that you’re testing. In my case, I’m trying to test a body validation, so it wouldn’t make sense to do a FactoryGirl.build where I specify the message body.

The second solution is to write a ‘before’ hook that lets the test assume that the subject is from a user, which will trigger the validation. Either method works well, but I opted to use the second option to practice a new technique.

Option 1

describe 'character limit for user-sent messages' do
  subject { FactoryGirl.build :message, contact: contact, status: 'sent' }
  it {should_validate_length_of(:body).is_at_least(1).is_at_most(420).on(:create) }
end

Option 2

describe 'character limit for user-sent messages' do
  before { allow(subject).to receive(:from_user?).and_return(true) }
  it {should_validate_length_of(:body).is_at_least(1).is_at_most(420).on(:create) }
end

Thoughts as a Developer Advocate

It has been almost a year since I graduated from DaVinci Coders, where I learned about Ruby and Rails. Since then, I have been a TA, worked as a consultant, and ultimately landed a job at TextUs. It hasn’t been easy, but it has been fun, exciting, and rewarding. Part of the reason I decided to pursue this career path was because of my brother, Nate, who encouraged me to take advantage of my technical savvy. Here are his thoughts on graduating from DevBootcamp in San Francisco, and how going to a bootcamp doesn’t necessarily mean you have to become a developer.

Nathan M Park – A Letter to bootcamp grads, 6 months as a Developer Advocate

Keeping track of Rails transactions

Ever heard of transaction_include_any_action?([:action])? You may never need to use this specialized part of ActiveRecord, but I thought I would shed some light on it anyway.

Recently, I was writing a module that helped incorporate several models into Elasticsearch. Concerns::Searchable, as I named it, helps create mappings for models, provides a base method for specialized searches, and also has an after_commit hook to index records. I first looked into ActiveRecord::Transactions to customize when a record is indexed into Elasticsearch.

module Concerns
  module Searchable
    extend ActiveSupport::Concern

    included do
      include Elasticsearch::Model
      after_commit :update_elasticsearch_index

      ...

      def update_elasticsearch_index
        if transaction_include_any_action?([:destroy]) || (transaction_include_any_action?([:update]) && self.archived?)
          ElasticsearchIndexWorker.perform_async(:delete, self.id)
        else
          ElasticsearchIndexWorker.perform_async(:index, self.id)
        end
      end
    end
  end
end

What I wanted was for Elasticsearch to delete records that were deleted in Postgres OR “archived”. That way, when a user goes to search for a record, they won’t accidentally get results from “archived” records that are being kept in Postgres for other purposes. Using ActiveRecord::Transactions helped me customize the logic for indexing after_commit.


NOTE: One small hiccup you may encounter is that the argument passed to the method has to be an array.

This is okay:

transaction_include_any_action?([:destroy, :update])

This is not:

transaction_include_any_action?(:destroy)

Adding foreign key constraints in Rails

Here’s a great article about adding foreign key constraints to handle cascading deletion of records. Basically, it’s a good idea to handle foreign key constraints through the database because Rails doesn’t always fire an object’s callbacks upon deletion. So if you have a dependent: :destroy callback set up, it might get skipped in some situations.

https://robots.thoughtbot.com/referential-integrity-with-foreign-keys

Indexing with the Elasticsearch Bulk API

If you’ve ever worked with the elasticsearch-rails gem, then you’ve likely used the import method to index your local Elasticsearch indexes. This is great for developing because it enables you to quickly index documents and work on the important stuff, like modifying search queries to your desired specification.

However, this technique doesn’t hold its water when you try to index a large number of documents on a production or staging environment. You should always work with Elasticsearch on a staging environment first to see how if your search queries behave as expect with several thousand documents before pushing to production. Similarly, you want to have a plan for indexing several million documents quickly.

The best strategy is to use Elasticsearch’s built in Bulk API, which enables you to index tens of thousands of documents in one request! I elected to use a Sidekiq worker to handle the creation of a bulk request with 1000 documents. This way, I can iterate over an entire table and batch my indexing into 1000 document-sized requests.

class ElasticsearchBulkIndexWorker
  include Sidekiq::Worker
  sidekiq_options retry: 5, queue: 'low'

  def client
    @client = Elasticsearch::Client.new host: CONFIG['ELASTICSEARCH_URL']
  end

  def perform(model, starting_index)
    klass = model.capitalize.constantize
    batch_for_bulk = []
    klass.where(id: starting_index..(starting_index+999)).each do |record|
      batch_for_bulk.push({ index: { _id: record.id, data: record.as_indexed_json } }) unless record.try(:archived)
    end
    klass.__elasticsearch__.client.bulk(
      index: "#{model.pluralize}_v1",
      type: model,
      body: batch_for_bulk
    )
  end
end

In order to iterate over an entire table, I simply created a rake task to handle counting and feeding in numbers to the worker. Notice that the ElasticsearchBulkIndexWorker takes a starting_index argument to handle batching. Also, it takes a string, which is the name of the model used to find the appropriate records and match them with the similarly-named index.

namespace :elasticsearch do
  task build_article_index: :environment do
    (1..Article.last.id).step(1000).each do |starting_index|
      ElasticsearchBulkIndexWorker.perform_async('article', starting_index)
    end
  end

  task build_comment_index: :environment do
    (1..Comment.last.id).step(1000).each do |starting_index|
      ElasticsearchBulkIndexWorker.perform_async('comment', starting_index)
    end
  end
end

This should help you index a high number of documents quickly. For more information, check out the Elasticsearch Bulk API documentation to figure out how to improve write consistency during your bulk indexing. Good luck!

GlueCon 2016

The past two days, I have been attending Glue Conference in Broomfield, CO. As they say, “Glue is a case study of what a conference is supposed to be.” Being a new software developer, it was the first tech conference I’ve ever attended, so I don’t have a gauge on whether or not it was better or worse than the average. That being said, I can say that I really enjoyed myself and learned A TON. Here are some of my reflections on the past 48 hours.


Pros
  1. You learn a LOT about emerging technologies and best practices for design and development. At my work, we have an API that our customers can interact with to integrate with their own software. We use Swagger to document our API, which works pretty well for our needs. One of the seminar tracks offered at GlueCon was all about APIs: design, implementation, maintenance, and deployment. What I took for granted at work was suddenly demystified; suddenly I could understand WHY our API was designed the way it is. Hearing people talk about big ideas helps you go beyond the HOW of developing.
  2. You get to have fun! I came into the conference with a very serious attitude. I told myself, I would be professional, network, and learn as much as I could. But everyone else seemed to have a different agenda. Obviously, people are there to learn and network, but they are also there to catch up with friends, make new ones, and just talk about what they’re interested in. I had a great time meeting tons of people, exchanging information, and just being able to share my excitement about being an up and coming developer. As an added bonus, you get a ton of free swag at these events – STICKERS GALORE!
  3. The keynote speakers are excellent. While some of the breakaway optional lectures were not the most exciting, all of the keynote speakers were quite good. While I wasn’t always able to follow them exactly (like the one about data encryption and authorization), I did get a sense of the scope of their ideas and the issues they were trying to address. More importantly, I learned how I fit in this grand ecosystem of development. It’s exciting when you begin to understand the role you play as a developer.
  4. You get to ask questions. Go to the booths and talk to people. Usually they want you to know all about their product, and if you don’t know what they’re talking about, they are willing to break it down and explain it to you. Before yesterday, I really had no idea what proxies were, but a gentleman at linkerd was willing to help me understand how their product fit in side-by-side with any existing app. Because there are so many people with so many specialties, you have the opportunity to be brave, ask questions, and learn so much from experts.

Cons
  1. You will feel like you know nothing. As a new developer, I was overwhelmed by the many facets of development that I still don’t understand. Most of the attendees were much older than me and had be in the tech industry for multiple decades. They would make jokes about the old days when they had to manage fires with their deployment and all I really know is how to use heroku to deploy my apps. That being said, it motivate me to work harder and try to be a more complete developer. I don’t necessarily need to be an expert in everything, but understanding my place will make me a stronger coder in the long run.
  2. Some people are not friendly. Just a couple of times, I mentioned that I was a Ruby on Rails developer and the person I was talking to straight up scoffed at me. Another time, someone tried to make a joke about how they felt sorry for my struggles. Look, I understand that maybe you don’t like Ruby or you might have opinions on my language of choice, but that doesn’t mean you have to be a dick. I found the best way to approach this behavior was to just smile, carry on, and try to leave as soon and politely as possible. I’d rather spend my time talking to people who are willing to help me grow as a coder and not just pigeon-hole me into some “this person is stupid” category.
  3. Some presentations are not good. There’s no getting around attending a few sub-par presentations. Not everyone can be a rock-star public speaker/developer/comedian/motivational speaker/coder/graphic designer. There was one talk I attended that was absolutely atrocious, and I tried my best to stay engaged and learn something. But I learned nothing because the speaker was too quiet and the content too dull. Since conferences are very long and exhausting, my advice would be to leave the room, take a walk, get some fresh air, and recharge for the next few hours.

Advice
  1. Go register for a conference! They are AWESOME.
  2. Look for scholarships online or ask your company for a subsidy. Maybe your boss is super nice and will support you if you’re going for a specific reason.
  3. Email people you want to meet before hand. Personally, I got a lot out of GlueCon specifically because I was able to meet with Lee Hinman from Elasticsearch, who gave me some great advice on implementing Elasticsearch into our Rails app at work.
  4. Go to the keynotes addresses. You may not understand everything the speakers are talking about, but you’ll definitely learn something.
  5. Take breaks. Go to the vendors and get some swag or go for a walk. It’s hard to sit through 6 straight hours of lectures.

Personally, I think GlueCon is amazing and I am definitely going to return next year. Hope to see you there!


2016-05-26 13.40.51

People start to trickle in and the exhibitors get fired up

2016-05-26 13.41.55

Joe Beda talks about authorization in the modern world

2016-05-26 13.01.31

I wont a Kindle Fire from Circonus!

Include vs. Extend…why not both?

If you don’t already know, there is a subtle difference between including and extending a module into one of your classes. include allows you to access instance methods while extend allows you to access class methods. See the example below for clarification:

module FooModule
  def foo
    puts 'foo'
  end
end

----------

class Bar
  include FooModule
end

Bar.new.foo => 'foo'
Bar.foo => NoMethodError: undefined method 'foo'

----------

class Bar
  extend FooModule
end

Bar.new.foo => NoMethodError: undefined method 'foo'
Bar.foo => 'foo'

So what if you want to create a module that will grant you access to both class and instance methods simultaneously? Luckily, using Rails’ ActiveSupport::Concern will help you separate your instance and class methods easily.

module FooModule
  extend ActiveSupport::Concern

  def foo
    puts 'foo'
  end

  module ClassMethods
    def baz
      puts 'baz'
    end
  end
end

----------

class Bar
  include FooModule
end

Bar.new.foo => 'foo'
Bar.foo => NoMethodError: undefined method 'foo'
Bar.new.baz => NoMethodError: undefined method 'foo'
Bar.baz => 'baz'