Blog
what did i learn today
Technology ruby on rails rspec
[RSPEC] Handling the MockExpectionError "does not implement"

I was writing a test for my helper which would, if the user is authorised to perform an action, show the actual link, and otherwise show the title or nothing (depending on what is wanted).

I make use of the vigilante gem which I should still write an article about, but this gem allows to store authorisations in the database and can be context-specific (e.g. you you can be assigned different roles for different "contexts", which could translate to organisations, projects, ... whatever applies to your setup). This gem adds a helper method is_allowed_to? to determine if one has the permissions to perform a certain given action.

When I wanted to mock this in my helper, I assumed a simple

     expect(helper).to receive(:is_allowed_to?) { true }

would suffice. However, I got a very strange error:

RSpec::Mocks::MockExpectationError: #<#<Class:0x00007fe2dc84e010>:0x00007fe29c83dc78 ....snip some very long things ... >> does not implement: is_allowed_to?

This is actually a great feature from Rspec: it checks if the thing you want to mock/stub actually exists on the original object, but in this case a helper (or a view) in Rails can access a lot of helper methods which are implicitly loaded.

This behaviour can be controller by setting the RSpec::Mocks.configuration.verify_partial_doubles to false. Of course I do not want to disable this for my entire test-suite, but just locally for the single test or single spec file.

So in my spec I temporarily disable the checking for the existence of doubles, as follows

require 'rails_helper'

RSpec.describe MapHelper do
  before(:all) do
    RSpec::Mocks.configuration.verify_partial_doubles = false
  end

  after(:all) do
    RSpec::Mocks.configuration.verify_partial_doubles = true
  end

  it "has the correct method" do
    expect(helper.respond_to?(:map_checked_link_to)).to eq(true)
  end

  context "map_checked_link_to" do
    context "a reachable feature" do
      before do
        @pipe = FactoryBot.create(:pipe)
      end
      it "renders a normal link" do
        allow(helper).to receive(:is_allowed_to?).and_return("XXX00")
        expect(helper.map_checked_link_to("Pijpstuk 123", @pipe, {})).to eq("<a href=\"/pipes/208\">Pijpstuk 123</a>")
      end
    end
    context "a blocked feature" do
      before do
      end
      it "does not render anything by default" do
        allow(helper).to receive(:is_allowed_to?) { false }
        expect(helper.map_checked_link_to("Pijpstuk 123", "", {})).to eq("Pijpstuk 123")
      end
      it "renders the title if so specified" do
        allow(helper).to receive(:is_allowed_to?) { false }
        expect(helper.map_checked_link_to("Pijpstuk 123", "", {}, true)).to eq("Pijpstuk 123")
      end
    end
  end

end

I also stumbled upon a PR implementing a method without_verifying_partial_doubles which takes a block which would do exactly the same. So one would be able to write

it "does something weird with mocks" do
  without_verifying_partial_doubles do
    ...
  end
end

But it did not work for me. Not sure if that is because the rspec version I am using in this project is too old, or the example I found was outdated.

More ...
Technology ruby on rails
[rails] Using ActionCable in production when using passenger+apache2

So on the server for work we use apache2+passenger to host all our rails servers. We now have about 10-ish, not too much, not all are used heavily, all are long-living, most are still rails 4. On my own server, with a similar setup, the rails versions vary from 4 to 6.

Recently I started a new project, and of course used rails 6. And I had the awesome idea to try out stimulus-reflex as this seems an interesting addition in the js front-end world. It really is a very interesting and different approach, and since the new/upcoming version 3.3 includes morphs, actually well-suited for our use-case which almost always involves a leaflet map (so we definitely need control over which area of the page to refresh). It works! Great. I will write in more detail about that at some other time.

So I had a working part of the website, enriched using stimulus/stimulus-reflex, and now I wanted to deploy it.

Only to realise ... passenger does not support ActionCable on apache2. ActionCable is one of those new tools I still need to investigate in detail, but yes, stimulus-reflex uses ActionCable to get a highly efficient channel between the browser and the server.

So now I am presented with two options:

  • we replace apache2 on our server with nginx. Frankly: this does not feel like a bad thing to do, however I am a bit hesitant to do this, since this seems to be a larger operation and will cause some down-time for our other applications.
  • use passenger-standalone and make apache2 proxy to it ---ha this seems interesting!

So I had to perform a few steps:

  • configure action-cable for deployment
  • install and run passenger standalone
  • change my apache2 config for my site

Configure ActionCable

Actually, this is amazing, but in development everything just works. Of course in production there are apparently a lot of options to deploy it.

I choose for the simplest setup: let my rails process host the websockets (and thus passenger), and use postgresql as the backend (I am not using redis yet).

So in my config/routes.rb I had to make sure we mount the ActionCable server

   mount ActionCable.server => '/cable'

In my config/environment/production.rb I specified our allowed domains:

  config.action_cable.allowed_request_origins = [ 'http://geotrax.be', /http:\/\/geotrax.*/ ]

And to use postgresql instead of redis is actually really simple (we were already using postgresql as our db backend). In your config/cable.yml just write

 production:
   adapter: postgresql 

Install passenger-standalone

I imagined this was easy, but it was only until I got all the steps correctly it worked. In short:

First I added the passenger gem to my Gemfile in the production block and move the puma to :development/:test

then create a Passengerfile.json file specifying the environment, port, user and instance_registry_dir

{
  // Run the app in a production environment. The default value is "development".
  "environment": "production",
  // Run Passenger on the given port. In this example, we use port 80,
  // the standard HTTP port.
  "port": 4000,
  // Tell Passenger to daemonize into the background.
  "daemonize": true,
  // Tell Passenger to run the app as the given user. Only has effect
  // if Passenger was started with root privileges.
  "user": "geotrax",
  "instance_registry_dir": "/srv/geotrax/tmp"
}

Somehow we still had to start passenger with the correct instance-registry-dir (specifying it in the Passengerfile did not work?). So we start passenger as follows

RAILS_ENV=production bundle exec passenger start --instance-registry-dir=/srv/geotrax/tmp

Then we have a running/waiting passenger instance. Now all that remains ...

Configure apache2 correctly

This was actually the hardest part. Also the passenger document just glosses over the subject. It is easy to write an apache site configuration that passes all http requests to passenger, but the websockets ... ?

To start you need the correct modules

sudo a2enmod proxy proxy_http proxy_wstunnel rewrite

(in my case this ment just adding the proxy_wstunnel mod, the other were already installed/used)

Then I edited my site.conf as follows:

<VirtualHost *:80>
    DocumentRoot /srv/geotrax/applications/geotrax/current/public

    ServerName  www.geotrax.be
    ServerAlias geotrax.be

    ErrorLog  /srv/geotrax/applications/geotrax/current/log/error.log
    CustomLog /srv/geotrax/applications/geotrax/current/log/access.log combined

    RewriteEngine on
    RewriteCond %{HTTP:Upgrade} websocket [NC]
    RewriteCond %{HTTP:Connection} upgrade [NC]
    RewriteRule /(.*) "ws://127.0.0.1:4000/$1" [P,L]

    ProxyPass / http://127.0.0.1:4000/
    ProxyPassReverse / http://127.0.0.1:4000/
    ProxyRequests Off

</VirtualHost>

Since we are using both http and websockets from the same domain, we are using the rewrite rules as explained in mod_proxy_wstunnel documentation

This took me a #$@!#$$%@#@ while to figure all out, I hope this helps someone to get started a little quicker (even if it is me the next time).

Next step: switch out apache2 and use nginx (but for now it is a little less urgent).

More ...
Technology bootstrap ruby on rails
[Bootstrap 4] Closing a modal programmatically

One of the UX patterns I like/prefer is to have some kind of in-place editing. So not to send the user to a completely new page or context, but edit or change a feature in place. One of the tools I frequently use for this is opening a modal.

This modal shows a mini-form, and will post the information to the server. There are two ways to handle this: we just post HTML and let the server decide what is rendered, or ... we post using js. In rails this is easy: we add remote: true.

For instance, I have a small form to resend a mail, but I want my users to overrule the email-address if they want to. The view looks something like this:

= link_to "(Re)send Email", '#', class: 'btn btn-large btn-info', 'data-toggle' => "modal", 'data-target' => "#resend-plan-request-#{plan_request.id}-email"

.modal.resend-plan-request.fade{id: "resend-plan-request-#{plan_request.id}-email"}
  .modal-dialog{role: "document"}
    .modal-content
      .modal-header
        %h4 Re(send) email for PlanRequest #{plan_request.external_id}
        %button.close{type: "button", 'data-dismiss' => "modal", 'aria-label' => "Close"}
      .modal-body
        = form_tag resend_admin_plan_request_path(plan_request), class: 'form', method: :post, remote: true do |f|
          .alert.alert-info.small
            This will allow either re-sending mails that clients/users claim have not received, or just for testing.

          .form-group
            = label_tag :emails, "Emails", class: 'form-label'
            .form-field-with-hint
              = text_field_tag :emails, plan_request.delivery_email, class: 'form-control', placeholder: 'Enter emails ...'
              %small.form-hint
                List the intended recipients, separate multiple emails using comma's e.g.
                %br
                email@abc.com, email2@def.com
          .form-group
            = submit_tag "Send Email", class: 'btn btn-primary', data: {disable_with: 'Sending ...'}
            = link_to 'Close', '#', class: "btn btn-secondary", 'data-dismiss' => "modal"

So for clarity: we have a small form in the modal, that will allow users to resend an email, and overrule the emails if wanted. We post the contents of the form to the resend action of the PlanRequestsController, which will actually do whatever is needed to send the email, and provide feedback to the user. I do not want to take the user to a new page, I just want to close the modal using js. According to the documentation this is simple, just do something like $(".modal").modal('hide');

So we write this inside resend.js.haml view

:plain
  toastr.success('#{@success_message}');
  $(".modal").modal('hide');

However I noticed that this leaves a .modal-backdrop div covering the entire screen, rendering a lot (or everything) unworkable. What seemed to be the case:

  • opening a modal using a button, adds 2 .modal-backdrop and closing a modal using the close-button(s) removes both
  • opening a modal using .modal('show') only adds a single .modal-backdrop, which is removed by the .modal('hide')

Problems:

  • controlling the modal using only buttons work
  • controlling the modal using only js code works
  • mixing (opening with button and closing with js) does not

I have two buttons that can close my modal: the x at the top, and a close button in the form, so I figured naively this bug (behaviour) was caused by the second close button for some reason. Unfortunately this is easy to verify (refute): if I remove the close button, it still adds two .modal-backdrop divs. So that was not a solution (and BTW I like to have a very visible close/cancel option).

It makes sense to close the modal in the same way it was opened, so either I change the way to open the modal, but I then tried if I could mimick "closing" by button by just triggering a click event on the close button(s) and then hoping it works as expected:

So do something like

    $("button[data-dismiss=modal]:visible").eq(0).click();

So to explain this:

  • first select all buttons that allow to close my modal: $("button[data-dismiss=modal]:visible") (select the visible just in case you have multiple modals defined in your html)
  • the added .eq(0) selects the first element out of that set
  • and then trigger the click event

Unfortunately this does not work either. During my numerous tests for a while it seemed to work, but I was just confused. Doh. This doesn't work either. I am guessing I just do not quite understand how bootstrap actually handles this, and (while I probably should) not really inclined to dive into their code to see what is wrong.

So I just went with:

:plain
  toastr.success('#{@success_message}');
  $(".modal").modal('hide');
  $(".modal-backdrop").remove();

Just close the modal, and if there are any pending/lingering .modal-backdrop divs, just remove those too.

More ...
Technology optimization ruby on rails
[RAILS] optimizing a slow request

We identified a single slow request in our moderation module: retrieving the json containing the entries to be moderated, feeding our react app.

So in short our datamodel looks as follows:

Class Topic 
  have_many :entries
  have_many :dynamic_attributes

Class Entry
  belongs_to :topic
  has_many :entry_values

Class EntryValue
  belongs_to :entry
  belongs_to :dynamic_attribute 

So in short: topics have a set of (dynamic) attributes that can be entered. An entry_value is the value entered for a dynamic_attribute and those are grouped in a complete entry.

In the moderation, our moderators verify that an entry (collection of entry-values) is appropriate, and have to option to edit or add missing information.

So in our controller we do something like

@entries = @q.result(distinct: true).page params[:page]

We are using ransack to filter/search on our entries. So we know we need entry-values, and possibly their dynamic attributes when building the json, so what is the best approach here? Use eager_load or includes? So what better way to decide than actually test this? So I temporarily changed the controller code as follows:

        preload_option = params[:pre].try(:to_i)
        if preload_option == 0
          @entries = @q.result(distinct: true).page params[:page]
        elsif preload_option == 1
          @entries = @q.result(distinct: true).eager_load(:entry_values).page params[:page]
        else
          @entries = @q.result(distinct: true).includes(:entry_values => :dynamic_attribute).page params[:page]
        end

Note:

  • I can only eager_load one level deep
  • with includes I can immediately fetch all dynamic attributes for the entry-values.

But which will prove to be more beneficial?

So then I ran a small benchmark script:

require 'benchmark'
require 'rest-client'

n = 10
Benchmark.bm do |x|
  x.report("normal ") do
    n.times { RestClient.get("http://admin.lvh.me:3013/moderation/projects/11/entries.json", {}) }
  end
  x.report("eager  ") do
    n.times { RestClient.get("http://admin.lvh.me:3013/moderation/projects/11/entries.json?pre=1", {}) }
  end
  x.report("include") do
    n.times { RestClient.get("http://admin.lvh.me:3013/moderation/projects/11/entries.json?pre=2", {}) }
  end
end

(note: for testing purposes I also disabled the need to authenticate, so I could easily fetch the jsons and time and compare)

This first run gave me the following results:

    normal   0.016362   0.005723   0.022085 ( 35.440171)
    eager    0.010036   0.004336   0.014372 ( 28.632490)
    include  0.012550   0.004061   0.016611 ( 29.173778) 

Ok. Not the kind of improvement I had hoped. Also nice to notice that eager_load in this case is more efficient than using the includes (which seemed a little counter-intuitive maybe).

I had recently changed a small part of the code, because in the moderation we also wanted to be able to edit fields that were not entered, and before we only had to retrieve entered values (:entry_values) so I presume that maybe there I fucked up the performance. Before we called entry.valid_entry_values which looked like

  def valid_entry_values
    entry_values.sorted.select do |ev|
      da = ev.dynamic_attribute
      da.attribute_type != 'item' || (da.attribute_type == 'item' && !ev.item_content_type.nil?)
    end
  end

and I replaced it with the following, adding empty entry-values to be filled in:


  def entry_values_with_empty
    result = []
    self.topic.dynamic_attributes.each do |da|
      ee = entry_values.find_by(dynamic_attribute_id: da.id)
      if ee.nil? || (da.attribute_type == 'item' && ee.item_content_type.nil?)
        ee = entry_values.build(dynamic_attribute: da)
      end
      result << ee
    end
    # check if we have entry-values not yet in the list
    # (e.g. from another topic when the entry was moved, and add those too)
    self.entry_values.each do |entry_value|
      if result.select{|ee| ee.id == entry_value.id}.count == 0
        result << entry_value
      end
    end
   
    result
  end

So what happens if we switch back to the old valid_entry_values : how does that change performance?

I ran my small benchmark script again, and got the following results:

    normal   0.015264   0.005280   0.020544 ( 33.283069)
    eager    0.009901   0.004359   0.014260 ( 17.145350)
    include  0.013153   0.004032   0.017185 ( 17.621856)

Wow! Now the eager_load or includes really seem to pay off. Also: almost the same speed improvement. Ok.

So if we check the entry_values_with_empty more closely, the implementation is somewhat naive: for each dynamic-attribute it will attempt to find the corresponding entry-value, except ... we use a query each time for each dynamic attribute, for each entry ... Mmmmmm. Let's see if we can improve this:

  def entry_values_with_empty
    result = []
    self.topic.dynamic_attributes.each do |da|
      ee = entry_values.detect{|ev| ev.dynamic_attribute_id == da.id}
      if ee.nil? || (da.attribute_type == 'item' && ee.item_content_type.nil?)
        ee = entry_values.build(dynamic_attribute: da)
      end
      result << ee
    end
    # check if we have entry-values not yet in the list
    # (e.g. from another topic when the entry was moved, and add those too)
    self.entry_values.each do |entry_value|
      if result.select{|ee| ee.id == entry_value.id}.count == 0
        result << entry_value
      end
    end

    result
  end

Notice: we only changed one line, replacing the find_by with a detect. This will, instead of launching a new query, iterate over the already retrieved array of entry_values. But does this make any difference?

Launching my small test script now returns the following:

    normal   0.016051   0.005418   0.021469 ( 16.448649)
    eager    0.009454   0.003858   0.013312 ( 22.479142)
    include  0.012419   0.003872   0.016291 ( 14.236868)

NICE! **fireworks** Not what I expected to see at all. A little baffled that the normal case is improved that much, and that the eager_load does not improve it (on the contrary). We have now found our optimal combination: improving the entry_values_with_empty and adding the includes will give the best performance.

Is this what you would have expected? Bottomline remains: it helps to measure (in Dutch we say: meten is weten which rhymes)

More ...
Technology ruby on rails rspec
[RSPEC] Cleaning up orphaned attachments when running specs

So when running the specs we also create a lot of fake attachments, but they are never cleaned up. Which is probably obvious, because we never actually destroy the models (containing the attachments), but truncate the database or rollback the transactions.

So I tried to find a way to 1) automatically/more easily clean up those dummy attachments, and 2) make sure it works when using parallel:specs. And over my different projects, where in some I use different gems to manage my attachments.

In one project, I am using paperclip and there I took the following approach. In the initializer config/initializers/paperclip.rb I wrote

  Paperclip::UriAdapter.register
  if Rails.env.production?
    Paperclip::Attachment.default_options.merge!(
      hash_secret: ENV.fetch('SECRET_KEY_BASE'),
      s3_protocol: :https,
      url: ':s3_domain_url',
      path: "/:class/:attachment/:id/:style/:hash.:extension",
      storage: :s3,
      s3_credentials: { .. }
    )
  elsif Rails.env.development?
    Paperclip::Attachment.default_options.merge!(
      url: "/system/:class/:attachment/:id/:style/:hash.:extension",
      hash_secret: Rails.application.credentials.secret_key_base
    )
  elsif Rails.env.test? || Rails.env.cucumber?
    Paperclip::Attachment.default_options.merge!(
      url: "/spec_#{ENV['TEST_ENV_NUMBER']}/:class/:attachment/:id/:style/:hash.:extension",
      hash_secret: Rails.application.credentials.secret_key_base
    )
  end

and then in rspec rails_helper.rb I can add the following piece of code

  config.after(:suite) do
    FileUtils.remove_dir(File.join(Rails.root, 'public', "spec_#{ENV['TEST_ENV_NUMBER']}"), true)
  end

In another projects I am using carrier_wave and there it is a little more difficult, but it amounts to the same approach. In CarrierWave we create different uploaders, and each have their own configuration. In my project I first iterate over all uploaders in my own code-base, and explicitly require one uploader from our own shared gem (between different projects). So we add an initializer config/carrierwave_clean_spec_attachments.rb (or whatever name you prefer) to override the path when in test mode:

if Rails.env.test? || Rails.env.cucumber?
  Dir["#{Rails.root}/app/uploaders/*.rb"].each { |file| require file }
  require 'document_uploader'

  CarrierWave::Uploader::Base.descendants.each do |klass|
    next if klass.anonymous?
    klass.class_eval do
      def cache_dir
        "#{Rails.root}/spec/support/uploads_#{ENV['TEST_ENV_NUMBER']}/tmp"
      end

      def store_dir
        "#{Rails.root}/spec/support/uploads_#{ENV['TEST_ENV_NUMBER']}/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
      end
    end
  end
end

and then in my rails_helper.rb I can then add the following statement:

config.after(:suite) do
  FileUtils.rm_rf(Dir["#{Rails.root}/spec/support/uploads_#{ENV['TEST_ENV_NUMBER']}"])
end

How do you do this? Do you use another gem for storage/attachments and how do you solve it? E.g. when using ActiveStorage ?

More ...
News wice_grid ruby on rails
[wice-grid] adding custom filters for a column

Let me quickly introduce WiceGrid, if you do not know yet: it is a super-gem that will allow you to easily show a list/grid of items, and allow easy filtering/searching/pagination.

For rails there is, afaik, no better alternative. There are some javascript/jquery driven dynamic grids, but for me the big advantage is that with WiceGrid all work is done server-side, which is ideal when handling large sets of data.

Since you can just render html in any column, we do for instance the following for our KLIP platform :

Screen Shot 2016-06-05 at 21.08.43

In our first we show our internal identifier, and the external identifier. In code this looks like this:

g.column name: 'Id', in_csv: false do |pr|
  render 'title_with_info', plan_request: pr
end

and the partial title_with_info looks like

%h4
  = plan_request.ident
%p.text-muted.small
  = plan_request.maprequest_id

But now the problem is: how can we, when filtering, automatically look for both fields? WiceGrid automatically handles one field, but not both. Luckily, WiceGrid allows us to define custom filter types. What we want is:

  • we want the filter to just look like a standard string field
  • we want to build a query which will search for ident or maprequest_id.

Adding your own custom filter types is not entirely clear in the documentation, I had to take a look at the code to fully understand it. So that's why I decided to write it in detail here.

It takes three steps:

  • define a class to create the correct filter (a conditions generator)
  • define a custom filter_type inside WiceGrid, using your custom class
  • use the class in the column definition

Create Conditions Generator

Inside lib/wice/columns add a new file called conditions_generator_column_plan_request_identifier.rb and add the following content:

module Wice
  module Columns
    class ConditionsGeneratorColumnPlanRequestIdentifier < ConditionsGeneratorColumn #:nodoc:

      def generate_conditions(table_alias, opts) #:nodoc:
        if opts.kind_of? String
          string_fragment = opts
          negation = ''
        elsif (opts.kind_of? Hash) && opts.has_key?(:v)
          string_fragment = opts[:v]
          negation = opts[:n] == '1' ? 'NOT' : ''
        else
          Wice.log "invalid parameters for the grid string filter - must be a string: #{opts.inspect} or a Hash with keys :v and :n"
          return false
        end
        if string_fragment.empty?
          return false
        end

        table_name = @column_wrapper.alias_or_table_name(table_alias)
        op = ::Wice.get_string_matching_operators(@column_wrapper.model)
        search_value = "%#{string_fragment}%"

        [
            " #{negation} (#{table_name}.ident #{op} ? OR #{table_name}.external_id #{op} ?)",
            search_value, search_value
        ]
      end
    end
  end
end

This class is actually almost copied from the standard column generator, except I generate a different condition at the end, where I compare with two fields with an OR operator. This way I will find results if either the ident or the external_id matches the search-value.

Define filter type in config

In config/wice_grid_config.rb add the following:

Wice::Defaults::ADDITIONAL_COLUMN_PROCESSORS = {
  plan_request_identifier_filter: ['ViewColumnString', 'Wice::Columns::ConditionsGeneratorColumnPlanRequestIdentifier']
}

We just use the standard ViewColumnString to just show us a string filter.

Use the filter_type in the column

To enable the filter in the column, we just have to write the following:

g.column name: 'Id', attribute: 'ident', filter_type: :plan_request_identifier_filter, in_csv: false do |pr|
  render 'title_with_info', plan_request: pr
end
More ...
News ruby gis oracle ruby on rails
[oracle] avoiding SLOW sdo_aggr_union

There is this recurring problem we have in GIS: getting road-segments and wanting to show complete roads. The naive approach would we to do something like the following:

insert into street_geoms
select ro.rd_ro_ident, ro.rd_ro_name, ro.com_code, ssdo_aggr_union(sdoaggrtype(rd.ro_geometry, 0.005)) as geom
from rd_road ro, rd_ro_sec ros
where ros.rd_ro_ident = ro.rd_ro_ident
group by ro.rd_ro_ident, ro.rd_ro_name, ro.com_code;

For good measure: we have 45.000+ roads, with a total of 230.000+ road segments. So when that query starts running and starts taking a long time, I started googling. Apparently there are two faster alternatives: SDO_AGGR_CONCAT_LINES and SDO_AGGR_SET_UNION. While the first was really quick, completed in minutes, the result was completely wrong (complete segments were missing). The second might be quicker, but it was really hard to get an idea about any progress, and if it would fail, everything should be lost (rolled back).

So I decided to write a little script, and issue a sql statement for each single road, allowing me to track progress and added restartibility. For each road I issued a statement like:

insert into street_geoms
select ro.rd_ro_ident, ro.rd_ro_name, ro.com_code, sdo_aggr_set_union(CAST(COLLECT(ros.rd_ros_geometry) AS mdsys.SDO_Geometry_Array),0.005) as geom
from rd_road ro, rd_ro_sec ros
where ros.rd_ro_ident = ro.rd_ro_ident
  and ro.rd_ro_ident = 1895101 
group by ro.rd_ro_ident, ro.rd_ro_name, ro.com_code;

I added some ruby code around it, to make sure it tracked the progress and calculated the remaining time, just to have an idea. The first "large" road it stumbled upon literally took hours. It only had to join 39 segments. A simple query learned I had 150+ roads with more segments, and a maximum of 125 segments in the database. I could not just simply ignore them :) So this was not going to work either.

Why would this be so hard? I just wanted to throw all linestrings together into one geometry. How could I do that? Querying the geometries was really easy, so what if I joined the geometries outside of oracle? And wouldn't that be hard? But there is a simple solution: convert the strings to WKT, and join all LINESTRING in a MULTILINESTRING. This would just be simple string manipulation. I can do that ;)

I had some hiccups with this approach: handling the long strings proved a bit akward (use CLOB instead) and I had to regularly call GC.start to make sure the open cursors were released. And I had to make sure not to build a string literal which was too long (ORA-06550).

But in the end I was able to join the road-sections for the 45.000 + roads in approx 1.5h, which is not blindingly fast, but faster than 1 single SDO_AGGR_SET_UNION operation :) :)

For reference you can see the full code:

class StreetGeom < ActiveRecord::Base
  self.primary_key = 'rd_ro_ident'
end

def format_time (t)
  t = t.to_i
  sec = t % 60
  min = (t / 60) % 60
  hour = t / 3600
  sprintf("% 3d:%02d:%02d", hour, min, sec)
end

def eta(count)
  if count == 0
    "ETA: --:--:--"
  else
    elapsed = Time.now - @start_time
    # eta = elapsed * @total / count - elapsed;
    eta = (elapsed / count) * (@total - count)

    sprintf("ETA: %s", format_time(eta))
  end
end

all_roads = Road.count
geoms_to_calculate = all_roads - StreetGeom.count
@total = geoms_to_calculate

puts "Joining geometries for #{all_roads} roads [still #{geoms_to_calculate} to do]"

cntr = 1
@start_time = Time.now

done = 0

Road.order(:rd_ro_ident).each do |road|
  street_count = StreetGeom.where(rd_ro_ident: road.rd_ro_ident).count
  print "\rConverting #{cntr}/#{all_roads} [#{eta(done)}] "
  if street_count == 0
    print "..."
    $stdout.flush

    ## get all geometries in WKT format
    get_geoms_sql = <<-SQL
      select sdo_cs.make_2d(ros.rd_ros_geometry).get_wkt() as wkt_geom from rd_ro_sec ros where ros.rd_ro_ident = #{road.rd_ro_ident}
    SQL

    cursor = Road.connection.execute(get_geoms_sql)

    line_strings = []

    while row = cursor.fetch
      line_string = row[0].read.to_s
      line_strings << line_string[10..-1]
    end

    insert_sql = <<-SQL
      DECLARE
        wkt_str clob;
      BEGIN
        wkt_str := 'MULTILINESTRING(#{line_strings.join(", ';\nwkt_str := wkt_str || '")})';
        insert into street_geoms(rd_ro_ident, name, com_code, geom)
        values (#{road.rd_ro_ident}, q'[#{road.rd_ro_name}]', '#{road.com_code}',
             sdo_util.from_wktgeometry(to_clob(wkt_str)) );
      END;
    SQL

    Road.connection.execute(insert_sql)
    done += 1
  else
    print "_"
  end

  cntr += 1

  # periodically cleanup GC so we release open cursors ...
  # to avoid ORA-1000 errors
  if (cntr % 50) == 0
    GC.start
  end
end

print "\n"
puts "\n\nDone!"

and I run this script in the rails environment as follows: rails runner lib\tasks\join_road_geometries.rb.

More ...
Technology ruby render_anywhere ruby on rails
Using render-anywhere gem with partials

Normally in rails, you can only render views inside of the controller. But what if you want to render a view somewhere else? For instance we wanted to generate xml-files using views. Haml can be used to describe xml just as well as plain html.

There is a gem called render_anywhere that allows just that. So how does this work, for example:

class Organisation < ActiveRecord::Base

  has_many :members

  include RenderAnywhere

  def to_xml
    render partial: "#{self.to_partial_path}", object: self, layout: 'my_xml_layout'
  end
end

We had a little problem when using partials though.

Normally if you type something like

= render @member

it will ask the partial path from the model (@member.to_partial_path), but somehow this always got prefixed with render_anywhere. The gem creates a dummy RenderingController in the RenderAnywhere namespace, so apparently it will look for the following view:

render_anywhere/members/member

In our case, I did not want to use the render_anywhere subfolder. It took me a while to figure out how to overrule this, but in essence it is pretty simple: rails uses the namespace of the rendering controller to prefix the path. Some deep googling proved that any controller has a method called _prefixes which lists all the prefixes for that class.

We can easily verify this in the rails console:

:001 > RenderAnywhere::RenderingController._prefixes
=> ["render_anywhere/rendering"]

So if we could overrule _prefixes to just return ["rendering"] ... Mmmmmm fork the code of render_anywhere? Or ...

There is another option: render_anywhere allows you to supply your own RenderingController and will use that instead if found in the context where the RenderAnywhere code is included.

So, if you write something like:

class Organisation < ActiveRecord::Base

  has_many :members

  include RenderAnywhere

  class RenderingController < RenderAnywhere::RenderingController

    def self._prefixes
      ["rendering"]
    end

  end

  def to_xml
    render partial: "#{self.to_partial_path}", object: self, layout: 'my_xml_layout'
  end
end

it will look for a view called members/member. Woot. To specify a different sub-folder you can adapt the _prefixes method as you wish :)

More ...
News schema_plus postgis ruby on rails
[rails] ignoring specific postgis view and tables in schema.rb

Developing rails websites with a geographic component we rely heavily on Postgis, so we use activerecord-postgis-adapter for the Postgis support, and I always use schema_plus because it allows me to define views. Until recently, I always had to use the structure.sql instead of the schema.rb because the geometric columns did not dump correctly.

But for a while now, activerecord-postgis-adapter handles this correctly and so we use the schema.rb file again. Only to discover a "new" error:

ActiveRecord::StatementInvalid: PG::DependentObjectsStillExist: ERROR: cannot drop view geography_columns because extension postgis requires it
HINT: You can drop extension postgis instead.
: DROP VIEW IF EXISTS "geography_columns"

Apparently specific Postgis views are also dumped in the schema file, and those views obviously cannot simply be re-created.

A very naive solution I kept using was to comment those create_view lines in our schema.rb file. But apparently there is a much better solution: you can configure which tables and views schema_plus should ignore.

So I added an initializer in /initializers/schema_dumper.rb with the following content:

ActiveRecord::SchemaDumper.ignore_tables = [
   "geography_columns", "geometry_columns", "spatial_ref_sys", "raster_columns", "raster_overviews"
]

And now my schema.rb is correct, and simple commands as rake db:setup or rake db:test:prepare just work. Hehe.

More ...
News ruby on rails
[rails] select distinct values of a column

The simplest way to select all distinct values of a column is, somewhat unintuitively:

Visit.uniq.pluck(:project)

this runs the query select distinct project from visits, and returns an array of strings. Exactly what you need, except ... I want it to be paginated. So we

Visit.uniq.pluck(:project).page(1)

... and that completely bombs: we now get an array of numbers?

So we try something else, and write:

Visit.select('distinct project')

which runs the good query, but returns an array of Visit's with only the project filled in. I can live with that. And then pagination (using the kaminari gem) is again as expected:

Visit.select('distinct project').page(params[:page])

Nice :)

More ...
News activerecord rails4 ruby on rails
[rails 4] add a reference to a table with another name

They default way in rails 4 to add foreign keys is simply

add_reference :people, :user

And this will add a column user_id to the people table, if the table exists. I have been looking in the rails code where this is handled, and it is really straightforward.

Note that standard rails does not really do anything for referential integrity, it creates the correct columns and an index if so specified.

But we use the schema_plus gem, for two reasons:

  • it does handle referential constraints correctly (on the database level)
  • it allows to specify the creation of views in a migration more cleanly

So, with schema_plus, if you write something like:

add_reference :people, :owner

and there is no owners table, this will give an error.

So instead you need to write :

add_reference :people, :owner, references: :users

This will correctly generate an owner_id which is a foreign key to users(id).

If you want to create a self-referential link, that is really easy. Just write

add_reference :people, :parent

This will create a self-referential link. Awesome :)

For completeness, the add_reference will add a reference columns to an already existing table. The same can be done when creating (or changing) a table with the following syntax:

create_table :people do |t| 
  t.references :owner, references: :users
  t.references :parent
end

So, in short, if you were not using the schema_plus gem already, start doing so now :)

More ...
News rails4 ruby on rails
How to clean assets in rails 4

Gentle reminder, do not forget, in rails 4

rake assets:clean

seems to work, but actually does nothing. That is not entirely true: it only cleans the old assets, leaving the three most recent versions of the assets intact. So it is a like a mild cleaning, a throw-away-the-garbage-clean, a bring-those-old-clothes-you-never-wear-to-the-thriftstore-clean.

But sometimes that does not cut it. Sometimes, don't ask me why, building my assets does not seem to work, my code is just not being picked up. And then you need to do use brute force cleaning (throw everything out). Run

rake assets:clobber

to actually clean the assets. The logic or meaning is lost on my me (clobber?), but this works.

More ...
Technology ruby windows ruby on rails
Run rails 2.3.18 using ruby 1.8.7 on windows server 2012

For my current job we have two 2.3.5 rails sites, of which I already succesfully upgraded one to rails 4. For the other we still need to start the migration, and we were asked to install new windows servers to run the rails servers on in the meantime (let's not digress why they choose windows, in a business environment Windows servers are still preferred, and let's be honest; they are easier to maintain and manage then *nix servers at first sight).

So whatever: I had to install the rails 2.3.5 on a new Windows 2012 server.

This proved problematic, since the new ruby 1.8.7 comes with the new rubygems, and this does not work nicely with rails 2.3.5.

So step 1: install the old ruby 1.8.7 (p302), the old gems and run the rails server. This worked.

But then I saw this one thing I really needed to improve. So I migrated the project to the latest 1.8.7 and rails 2.3.18, use bundler for gem dependencies. On my development box this worked like a charm (Macbook Pro). So then I deployed this back on the server, and then the following happened:

C:/Ruby187/lib/ruby/1.8/pathname.rb:290:in `[]': no implicit conversion from nil to integer (TypeError)
    from C:/Ruby187/lib/ruby/1.8/pathname.rb:290:in `chop_basename'
    from C:/Ruby187/lib/ruby/1.8/pathname.rb:343:in `cleanpath_aggressive'
    from C:/Ruby187/lib/ruby/1.8/pathname.rb:331:in `cleanpath'
    from C:/Ruby187/lib/ruby/gems/1.8/gems/rails-2.3.18/lib/rails/rack/log_tailer.rb:9:in `initialize'
    from C:/Ruby187/lib/ruby/gems/1.8/gems/rack-1.1.6/lib/rack/builder.rb:54:in `new'
    from C:/Ruby187/lib/ruby/gems/1.8/gems/rack-1.1.6/lib/rack/builder.rb:54:in `use'
    from C:/Ruby187/lib/ruby/gems/1.8/gems/rack-1.1.6/lib/rack/builder.rb:73:in `call'
    from C:/Ruby187/lib/ruby/gems/1.8/gems/rack-1.1.6/lib/rack/builder.rb:73:in `to_app'
    from C:/Ruby187/lib/ruby/gems/1.8/gems/rack-1.1.6/lib/rack/builder.rb:71:in `inject'
    from C:/Ruby187/lib/ruby/gems/1.8/gems/rack-1.1.6/lib/rack/builder.rb:73:in `each'
    from C:/Ruby187/lib/ruby/gems/1.8/gems/rack-1.1.6/lib/rack/builder.rb:73:in `inject'
    from C:/Ruby187/lib/ruby/gems/1.8/gems/rack-1.1.6/lib/rack/builder.rb:73:in `to_app'
    from C:/Ruby187/lib/ruby/gems/1.8/gems/rails-2.3.18/lib/commands/server.rb:95
    from script/server:3:in `require'
    from script/server:3

What was going on here? Apparently in the rails/rack/logtailer.rb is the following code:

class LogTailer
  def initialize(app, log = nil)
    @app = app

    path = Pathname.new(log || "#{::File.expand_path(Rails.root)}/log/#{Rails.env}.log").cleanpath

And the cleanpath is somehow crashing. I tried googling this, to no avail.

Of course:

  • rails 2.3.18: who uses that still?
  • ruby 1.8.7 is deprecated
  • and deploying on windows servers

So I was on my own. I tracked the error down to the cleanpath_agressive which called the chop_basename recursively to remove superfluous . and .. redirections.

I am guessing the problem is that on windows, a path starts with a drive-letter, like D:\ or C:\ which messes up the ending of the cleanpath_aggressive loop.

Instead of really diving in, the path handed down to cleanpath in my case, did not need any cleaning, and furthermore, an uncleaned path would still work.

So I added an initializer config\cleanpath_bug_fix.rb with the following code:

if RUBY_PLATFORM =~ /mingw/
  # on windows the Pathname.cleanpath crashes on windows paths, instead of fixing it thoroughly
  # just ignore it
  class Pathname
    def cleanpath_aggressive
      self
    end
  end
end

Now my rails 2.3.18, using ruby 1.8.7p374 runs on Windows Server 2012R2. Woot ;)

More ...
Technology test coverage testing ruby on rails
complete coverage with SimpleCov

When using SimpleCov in a very ill-covered project, I got amazingly good results: SimpleCov just did not count not-covered files. So files that were never used in our test-suite, were just simply being ignored. While I understand that approach, it did not feel good. I want to measure absolute progress, and I want to know how badly it really is.

So, on a mission to count all uncovered files with simplecov, I encountered an issue in their github repo. In itself it contained/mentioned two solutions:

  • in a three year old comment, a solution to set a starting base-line, and merge it after the tests. Unfortunately, it did not work completely: all files were added, but with complete coverage. Doh.
  • a pull request, claiming to fix it: while it did add all files, files that had before 100% coverage, where now no longer.

Let's get technical. The first solution added a baseline, with for all lines the value nil. The second added the value 0 for each line. The value nil is what SimpleCov uses internally to count for a never line: a line that never matters. Which is either an empty line, a comment line, or begin and end of a class.

The zero is line that is not covered (a 1 is a line that was covered). When merging lines for files that had coverage, I assume the base-line 0 takes precedence over the coverage-calculated nil and so we end up with a non-covered line in the merged result. Bummer.

So I added my spec_helper.rb as follows:

if ENV["COVERAGE"]
  require 'simplecov'
  SimpleCov.start 'rails'
  SimpleCov.coverage_dir 'coverage/rspec'

  all_files = Dir['**/*.rb']
  base_result = {}
  all_files.each do |file|
    absolute = File::expand_path(file)
    lines = File.readlines(absolute, :encoding => 'UTF-8')
    base_result[absolute] = lines.map do |l|
      l.strip!
      l.empty? || l =~ /^end$/ || l[0] == '#' ? nil : 0
    end
  end

  SimpleCov.at_exit do
    coverage_result = Coverage.result
    covered_files = coverage_result.keys
    covered_files.each do |covered_file|
      base_result.delete(covered_file)
    end
    merged = SimpleCov::Result.new(coverage_result).original_result.merge_resultset(base_result)
    result = SimpleCov::Result.new(merged)
    result.format!
  end
end

So, before the test runs, I create a "baseline" containing for all possible ruby files (might need to filter that later), a nil if the line is empty or the "ending end" of a file, and a zero otherwise. After the tests have run, I remove the covered files from the "baseline", and those are then merged with the result that get the final result.

Now maybe try to translate that into a pull request :)

More ...
News netzke ruby on rails
[netzke] make action buttons toggle

I am currently converting my GIS application that needs two browser windows (one for the map, and one for the administrative data and functions), to a single window application using the netzke gems (which in turn rely on Sencha Ext js).

The netzke gems are extremely powerful, but there is a bit of learning curve for me, since both netzke and ext js are new. But netzke makes a lot of things a lot easier. It is not really rails-like, since it abstracts a lot away, and I am not yet completely convinced of the approach. But for my current goals it is ok.

I have a panel containing an openlayers map, and a toolbar with action buttons. The action buttons need to indicate state (drawing/measuring). So when clicked they have to be in a "pressed" state, and start with the action.

Although I found no example of this, achieving it was pretty easy. I defined my action buttons as follows:

action :measure_line do |c|
  c.id = "measure_line"
  c.icon = :ruler
  c.text = ""
end

action :measure_area do |c|
  c.id = "measure_area"
  c.icon = :surface
  c.text = ""
end

js_configure do |c|
  c.layout = :fit
  c.mixin
end

def configure(c)
  c.desc = "Contains the openlayers map"
  c.title = "Map"
  c.tbar = [:draw_point, :draw_line, :draw_polygon, :measure_line, :measure_area, :navigation_history]
  c.body_padding = 5
end

I am only showing the actions for :measure_line and :measure_area because these are now the relevant ones I want to show. I need to give explicit id to the buttons, so that I can use Ext.ComponentManager.get to find them later on.

Nice thing to know is that the action handler by default receives the pressed button (yes!). This will allow us to just toggle it. So your mixed in javascript will look this:

{
    onMeasureLine: function(btn) {
      Map.toggleMeasuring(btn, 'line');
    },
    onMeasureArea: function(btn) {
      Map.toggleMeasuring(btn, 'polygon');
    },
}

Because I prefer to write coffeescript (not found how I can do that for the netzke mixins), I have a class Map in map.js.coffee containing all map-related functions. toggleMeasuring needs the button and the measurement-action to activate/deactivate

toggleMeasuring: (button, measurement_type) ->
  button.toggle()
  if 'pressed' in button.uiCls
    @measureControls[measurement_type].activate()
  else
    @measureControls[measurement_type].deactivate()

  # make sure the other measuring is automatically switched off
  for key of @measureControls
    unless key == measurement_type
      @measureControls[key].deactivate()
      Ext.ComponentManager.get("measure_#{key}").toggle(false)

That is about the gist of it. I will be sharing more of my netzke experience and code soon. For the moment still impressed :)

More ...
News heroku pdf ruby on rails
generating pdf on heroku

I was investigating ways to generate pdf's in Ruby on Rails, but I had one enormous constraint: it had to deploy on heroku.

There are two very different ways to generate pdf's in ruby:

  • use prawn: it is pure ruby, very powerful. It has it's own DSL, that unleaches all the power of building a PDF, but at the same time: it seems to be very hard and tedious.
  • use some sort of HTML to PDF conversion. In ruby there exists two gems: wicked_pdf and PDFKit, both use wkhtmltopdf under the covers. I dreamed of having a view magically converted to PDF.

I went for the second option. Furthermore, I choose wicked_pdf over PDFKit, because I felt the rails integration was better. It allowed me to just render a view which would automatically be downloaded as a PDF.

Setting up wicked_pdf in Rails to run on heroku

Luckily, getting it running on heroku proved to be incredibly easy: just including the correct gem with the binaries that work on heroku.

In my Gemfile I added the following:

 gem "wicked_pdf"
 gem "wkhtmltopdf-heroku", :git => 'git://github.com/camdez/wkhtmltopdf-heroku.git' 

And, then, inside a view you want to render as pdf, write something like

 respond_to do |format|
   format.js 
   format.pdf { 
     render :pdf => "show", :header => { :font_size => '8', :right => '[page] of [toPage]' }, :footer => {:font_size => '8', :right => 'Generated by jottinx.com' } 
   } 
 end

Then, you will still have to create the view, show.pdf.erb. Just make sure your view renders HTML and it will be converted to PDF correctly. That is just awesome.

Hope this helps.

More ...
Technology capybara cucumber ruby on rails
testing drag and drop of jQuery UI sortable with cucumber and capybara

The Problem

For jottinx I wrote a small piece of code that allowed to sort items using drag and drop. Of course, after writing it and making sure it works (manually), I want to make sure it keeps working. So I add a test, using cucumber.

My scenario actually looks pretty straightforward:

 @javascript
 Scenario: Dragging a book
   Given I authenticate as a "default user"
   And I add a new book with title "a-new-book"
   And I add a new book with title "another-book"
   And I add a new book with title "one-last-book"
   And I drag book "one-last-book" to the top
   Then book "one-last-book" is at the top of the list

The difficult bit was: how do I implement the dragging?.

Actually it seemed straightforward, because capybara has a method called drag_to. So I implemented the step like this:

 When /^I drag book "([^"]\*)" to the top$/ do |book_title|
    drop_place = page.find(:css, 'ul.sortable-books li:first')
    page.find(:xpath, "//a[@href='##{book_title.parameterize}']").drag_to(drop_place)
 end

But, unfortunately, this did not work. I googled around a bit and found the following two similar questions:

The short conclusion: it does not work, and it is a combination of how jquery implemented the sortable element, and the fact that the selenium driver does not support it yet. So, refraining to using the selenium driver directly does not help either. After some more googling, I found a similar question on stackoverflow, and there I found the solution.

The solution

Enter jquery.simulate.drag-sortable.js. It is a script that will allow you to simulate dragging in a sortable object by issuing a simple javascript command:

 // drag item down one position in the list
 $('#itemToDrag').simulateDragSortable({ move: 1 });

If move parameter is negative, it will move up. And down if positive. If you include the js inside your project, you can easily test that out inside your Chrome console. It just works. Awesome piece of work. To use that in a step-definition, just write:

 When /^I drag book "([^"]\*)" to the top$/ do |book_title|
    page.execute_script %Q{ 
      $('.sortable-books li:last').simulateDragSortable({move: -4}); 
    }
 end

Hope this helps :)

More ...
News cocoon nested model form rails3 ruby on rails
a guide to doing nested model forms in rails (3.1)

One recurring problem when doing Ruby on Rails development is a nested model form.

Nested Model Form: a single form that contains multiple, nested models.

For example a project with its tasks. A nested model form will allow you to create or edit, in 1 form, the project and each of its tasks.

With the use of cocoon, this post will describe how to create a nested model form for some (all?) of the possible relations.

All these examples start from the project, so we will just start with a dummy project as follows:

rails g scaffold Project name:string

That's all we need to get started. In these examples i am using simple_form and slim (which looks a lot like haml, but is much faster).

The simple :has_many

The simple :has_many is the most occurring relation (unfortunately no scientific data to back that up). The simple :has_many is a relation where the child cannot exist without the parent, and where each child can have only one parent. A typical example is the project and his tasks. Each task is unique, each task only exists because of the project. If the project is gone, so are the tasks.

First the models:

class Project < ActiveRecord::Base 
  has_many :tasks 
  accepts_nested_attributes_for :tasks, :reject_if => :all_blank, :allow_destroy => true
end 

class Task < ActiveRecord::Base 
  belongs_to :project, inverse_of: :tasks 
end

The accepts_nested_attributes_for method makes sure that when posting the project-data, it will accept the attributes for the nested model.

Secondly we add the views. Inside the project _form.html.slim we add

  #tasks
    = f.simple_fields_for :tasks do |task|
      = render 'task_fields', :f => task
    .links
      = link_to_add_association 'add task', f, :tasks

Create a partial called projects_task_fields.html.slim which looks like:

.nested-fields
  = f.input :name
  = f.input :description
  = f.input :done, :as => :boolean
  = link_to_remove_association "remove task", f

The nested :has_many

Actually this is a simple extension of the previous example. For instance, a project with tasks, and each tasks has sub-tasks.

Our models:

  class Task < ActiveRecord::Base 
    has_many :sub_tasks 
    accepts_nested_attributes_for :sub_tasks, :reject_if => :all_blank, :allow_destroy => true 
end 

class SubTask < ActiveRecord::Base 
end

Edit the projects_task_fields.html.slim :

.nested-fields 
  = f.input :name 
  = f.input :description 
  = f.input :done, :as => :boolean 
  .sub_tasks 
    = f.simple_fields_for :sub_tasks do |sub_task|
    = render 'sub_task_fields', :f => sub_task 
    .links 
      = link_to_add_association 'add sub-task', f, :sub_tasks
  = link_to_remove_association "remove task", f

and add the view projects_sub_tasks.html.slim (which bears a lot of resemblance to the previous tasks-partial):

.nested-fields 
  = f.input :name 
  = f.input :description 
  = link_to_remove_association "remove sub-task", f

The look-up or create :belongs_to

A simple :belongs_to would mean that the parent would already exist, and a simple look-up (dropdown list) would suffice. If the parent is always unique, it can be solved in the same way as the :has_many. So, let's describe a solution for the case where we either select the item from a list, or create a new one.

Our project has an owner (which is a person):

    class Project < ActiveRecord::Base
      belongs_to :owner, :class_name => 'Person'
      accepts_nested_attributes_for :owner, :reject_if => :all_blank
    end
    class Person < ActiveRecord::Base
    end

inside the _projects/_form.html.slim we add the following:

    #owner
      #owner_from_list
        = f.association :owner, :collection => Person.all(:order => 'name'), :prompt => 'Choose an existing owner'
      = link_to_add_association 'add a new person as owner', f, :owner

Here we use the built-in way from simple_form to select an association from a look-up list: f.association. But secondly, we place the link_to_add_association that will render the partial projects/_owner_fields.html.slim and create a new Person and link to him.

The partial itself is pretty straightforward:

    .nested-fields
      = f.input :name
      = f.input :role
      = f.input :description
      = link_to_remove_association "remove owner", f

Now if you implement this, the form is functioning but not really user-friendly.

What we want is that when clicking the add a new person as owner-link, that the drop-down box and the link itself disappear. When we choose to remove the to-be created owner, that the drop-down and add-link reappear.

So we need to add a bit of extra javascript.

Open up app\javascripts\projects.js and add:

    $(document).ready(function() {
      $("#owner a.add_fields").
        data("association-insertion-position", 'before').
        data("association-insertion-node", 'this');

      $('#owner').bind('insertion-callback',
         function() {
           $("#owner_from_list").hide();
           $("#owner a.add_fields").hide();
         });
      $('#owner').bind("removal-callback",
         function() {
           $("#owner_from_list").show();
           $("#owner a.add_fields").show();
         });
    });

The first two lines control where the new partial (the new owner) will appear.

Upon insertion or removal, cocoon will trigger callbacks that are defined on the parent-container of the add-link. The last lines add those callbacks, and these will make sure that the link and dropdownbox will be hidden or shown again.

The :has_many :through relation

A classic example of this relation are tags. Something has tags, those can be chosen from an exisiting list of tags, one or more, or one could create new tags. This relation is, again, an extension of the previous solution.

This solution is a bit more complicated, because there is a bit more javascript involved here.

The models:

    class Project < ActiveRecord::Base
      has_many :project_tags
      has_many :tags, :through => :project_tags, :class_name => 'Tag'

      accepts_nested_attributes_for :tags
      accepts_nested_attributes_for :project_tags
    end
 
    class ProjectTag < ActiveRecord::Base
      belongs_to :tag
      belongs_to :project

      accepts_nested_attributes_for :tag, :reject_if => :all_blank
    end

    class Tag < ActiveRecord::Base
    end

inside the projects/_form.html.slim add:

    #tags
      = f.simple_fields_for :project_tags do |project_tag|
        = render 'project_tag_fields', :f => project_tag
      = link_to_add_association 'add a tag', f, :project_tags

Create a file projects/_project_tag_fields.html.slim containing:

.nested-fields.project-tag-fields
  #tag_from_list
    = f.association :tag, :collection => Tag.all(:order => 'name'), :prompt => 'Choose an existing tag'
  = link_to_add_association 'or create a new tag', f, :tag
  = link_to_remove_association "remove tag", f

And create another file projects/_tag_fields.html.slim:

.nested-fields
  = f.input :name, :hint => 'how should it be tagged'

When entering all this, it should be working, but it looks a bit confusing. The dropdown box does not disappear (as before). So we need the following bit of javascript to projects.js:

    $("#tags a.add_fields").
      data("association-insertion-position", 'before').
      data("association-insertion-node", 'this');

    $('#tags').on('cocoon:after-insert',
         function() {
             $(".project-tag-fields a.add_fields").
                 data("association-insertion-position", 'before').
                 data("association-insertion-node", 'this');
             $('.project-tag-fields').on('cocoon:after-insert',
                  function() {
                    $(this).children("#tag_from_list").remove();
                    $(this).children("a.add_fields").hide();
                  });
         });

While this javascript is doing almost the same as with the :belongs_to it is a bit more complicated, because the tag-partial is not yet inside in the page, so neither is the parent-div. Once that is clear, it is actually quite obvious what it does (I hope). Upon insertion of a new (project)tag, it also adds the callbacks for the insertion and removal of a new tag.

Example Code

All these examples are demonstrated in a working project: cocoon-simple-form-demo. If you can think of more relations you would like to see solved, or see an example of, please let me know.

Hope this helps.

More ...
Technology rails31 ruby on rails
rails 3.1: creating a new application (fast!)

Take the following steps, by preference install this in a specific gemset [ruby] rvm gemset create rails31 rvm gemset use rails31 gem install bundler gem install rails --pre [/ruby] These steps will create a new gemset and start using it, then install bundler and rails 3.1 (which is still pre-release at the time of writing). Now in rails 3.1 there is a new requirement: you will need a javascript runtime. You can see the full list here. If you are on linux and using straight ruby then therubyracer is just fine. Type [ruby] gem install therubyracer [/ruby] Creating a new rails-project always poses the same problem: creating a new project, with your set of gems, which all have to installed, generators need to be ran ... Rails Wizard solves that. [ruby] gem install rails_wizard [/ruby] Type rails_wizard list to see all possible templates. To create a new rails 3.1 project using jquery, rspec, haml, devise you just need to write: [ruby] rails_wizard new your_new_application_name -r jquery rspec haml devise [/ruby] To go full out on options, you could write: [ruby] rails_wizard new your_new_application_name -r jquery rspec haml devise sass settingslogic git [/ruby]

Note

If you are using a specific gemset for your project, please remember to add your javascript runtime (e.g. I used therubyracer) to your Gemfile. ... and you are ready to go! Happy coding :)

More ...
News ci teamcity ruby on rails
configuring TeamCity to run rails projects

In our team we are very happy users of RubyMine, by JetBrains. Now JetBrains also has a continuous integration server, called TeamCity, and it is also capable to run rails rake tasks. Installing TeamCity is close to a non-operation, as described on their website. Just download the package, extract, and run bin/runAll.sh start. Then you can browse to http://localhost:8111 and you are up and running. Unfortunately, getting the build to actually run was a bit more complicated in our case. Our prerequisets:

  • we do not check in any configuration file (like config.yml, database.yml, ...) so each developer can have their own settings.
  • we use bundler to manage our gem dependencies We use the rake runner, but as you might know: rake tasks will not run unless the bundle is up to date. Luckily we are not the first to encounter that problem, and the suggested solution is actually quite simple, although a bit backward: use a custom rakefile, that will first run the bundle install, then require the actual Rakefile from which you can run the tasks. To create the log folder, the css files (we use sass), the config-files, i had to create specific rake-tasks. We also had a problem that our database was always somehow corrupt. Upon thorough investigation, i was able to pin-point this on the rake db:test:prepare task, that actually does not load the database from the schema.rb, but tries to clone the development database! On our continous integration server there is no development database! So i cleared the db:test:prepare task. Our custom rakefile then looked as follows: [ruby] directory "log" task 'copy_rakefile' do cp 'Rakefile', 't_rakefile.rb', {:verbose => true} end task :bundle_install do sh 'bundle install' end task :sass do Sass::Plugin.update_stylesheets end task :default => [:bundle_install, 'copy_rakefile', 'log'] do RAILS_ENV = ENV['RAILS_ENV'] = 'test' require File.dirname(__FILE__) + '/t_rakefile' # launch rake tasks from the original Rakefile Rake::Task["log:clear"].invoke Rake::Task["teamcity:init_config"].invoke Rake::Task['db:test:prepare'].clear Rake::Task["db:reset"].invoke Rake::Task["environment"].invoke Rake::Task['sass'].invoke Rake::Task["test"].invoke Rake::Task["spec"].invoke Rake::Task["cucumber"].invoke end [/ruby] We had to copy the Rakefile, so we could later require it as a straight ruby-file. Note that to disable the db:test:prepare i used the following code: [ruby] Rake::Task['db:test:prepare'].clear [/ruby] Simple, isn't it :) It's also only cleared when running the teamcity:build, just as we want it. Now we got this working, it is time to enjoy the goodies TeamCity offers us, like spreading over different build agents.
More ...