I develop my website using a MacBook Pro retina, and deploy on windows. And I noticed that my Macbook Pro is a lot faster than the standard GeoServer install. I use Geoserver only to serve the WMS layers, vector data which is stored in postgis.

So I needed to tune GeoServer on Windows for optimal performance. I googled around, and found I needed to make the following changes:

• make sure jvm runs in -server mode
• make sure jvm is allocated enough memory
• optimise jvm settings
• install native JAI/ImageIO binaries
• switch to production logging

I used the standard Geoserver binary install, so to tune the jvm settings, you have to edit c:\program files\Geoserver xxx\wrapper\wrapper.conf and add the following lines:

# Java Additional Parameters
wrapper.java.additional.1=-Djetty.home=.
wrapper.java.additional.2=-DGEOSERVER_DATA_DIR="%GEOSERVER_DATA_DIR%"
wrapper.java.additional.3=-server 
wrapper.java.additional.4=-Xmx2048M -Xms2048m 
wrapper.java.additional.5=-XX:SoftRefLRUPolicyMSPerMB=36000 
wrapper.java.additional.6=-XX:MaxPermSize=128m 
wrapper.java.additional.7=-XX:NewRatio=2

to be able to run the jvm in server mode, I had to copy $JAVA_HOME\bin\client to $JAVA_HOME\bin\server which feels like an awesome hack.

But Geoserver does seem a lot quicker.

I read the wms shootout benchmarks (2011) and one of the results showed that MapServer was a lot quicker on linux (vs. windows).

So that has got me wondering though, what possible other things I could do to improve performance.

E.g.

• deploy geoserver on linux vs windows?
• switch containers: jetty vs tomcat vs jboss, or doesn't that make much of a difference?
• or how hard would it be to switch to a "quicker" wms, e.g. mapnik/mapserver

So since none of this questions I can find answered somewhere on the web (at least not easily, and i asked), I will probably be doing some benchmarks for myself soon.

I am currently converting my GIS application that needs two browser windows (one for the map, and one for the administrative data and functions), to a single window application using the netzke gems (which in turn rely on Sencha Ext js).

The netzke gems are extremely powerful, but there is a bit of learning curve for me, since both netzke and ext js are new. But netzke makes a lot of things a lot easier. It is not really rails-like, since it abstracts a lot away, and I am not yet completely convinced of the approach. But for my current goals it is ok.

I have a panel containing an openlayers map, and a toolbar with action buttons. The action buttons need to indicate state (drawing/measuring). So when clicked they have to be in a "pressed" state, and start with the action.

Although I found no example of this, achieving it was pretty easy. I defined my action buttons as follows:

action :measure_line do |c|
  c.id = "measure_line"
  c.icon = :ruler
  c.text = ""
end

action :measure_area do |c|
  c.id = "measure_area"
  c.icon = :surface
  c.text = ""
end

js_configure do |c|
  c.layout = :fit
  c.mixin
end

def configure(c)
  c.desc = "Contains the openlayers map"
  c.title = "Map"
  c.tbar = [:draw_point, :draw_line, :draw_polygon, :measure_line, :measure_area, :navigation_history]
  c.body_padding = 5
end

I am only showing the actions for :measure_line and :measure_area because these are now the relevant ones I want to show. I need to give explicit id to the buttons, so that I can use Ext.ComponentManager.get to find them later on.

Nice thing to know is that the action handler by default receives the pressed button (yes!). This will allow us to just toggle it. So your mixed in javascript will look this:

{
    onMeasureLine: function(btn) {
      Map.toggleMeasuring(btn, 'line');
    },
    onMeasureArea: function(btn) {
      Map.toggleMeasuring(btn, 'polygon');
    },
}

Because I prefer to write coffeescript (not found how I can do that for the netzke mixins), I have a class Map in map.js.coffee containing all map-related functions. toggleMeasuring needs the button and the measurement-action to activate/deactivate

toggleMeasuring: (button, measurement_type) ->
  button.toggle()
  if 'pressed' in button.uiCls
    @measureControls[measurement_type].activate()
  else
    @measureControls[measurement_type].deactivate()

  # make sure the other measuring is automatically switched off
  for key of @measureControls
    unless key == measurement_type
      @measureControls[key].deactivate()
      Ext.ComponentManager.get("measure_#{key}").toggle(false)

That is about the gist of it. I will be sharing more of my netzke experience and code soon. For the moment still impressed :)

I am currently converting an Oracle database to Postgis. Instead of blindingly copying the data model, I am also checking which columns are actually really used, and drop those that are never used.

In most tables it is pretty easy, I can do a quick visual check and then count that one column that seems to be zero all the time. But we have a few tables with 30-50 columns.

For such a table there is also an easy way:

SELECT t.column_name
FROM user_tab_columns t
WHERE t.nullable = 'Y'
    AND t.table_name = 'YOUR_TABLE_NAME_HERE'
    AND t.num_distinct = 0

Mind you, for this to work, your database must have gathered the statistics (if you haven't done this before, this will also help your performance).

BEGIN
  DBMS_STATS.gather_database_stats();
END;

Yowza. Due to some devine intervention I am now responsible to upgrade three rails sites. I started developing them in 2009, but haven't touched then since mid 2010. The persons who took over have not kept it up to date, nor (from their code) were they really good ruby programmers :)

So I have the daunting task to bring those projects into the present :)

Remember, 2010, rails 2.3.5? That seems like ages ago :) There was

• no bundler
• no rvm (actually, still isn't since they deploy on windows --aaaaarggh since then I develop on ubuntu and mac)
• vendor/plugins instead of gems
• no asset pipeline
• old routes

At first I couldn't even get the correct set of gems together to get the rails site running correctly.

So the steps I took to get it running (on my mac first) :

• create a new branch
• use rvm to switch to start using 1.8.7 and a new empty gemset
• gem install rails (which apparently installs the correct 2.3.5 version by itself --impressed :)
• install rails_update plugin (script/plugin)
• my current rake version is not compatible, luckily if you type rake _0.8.7_ instead of the normal rake it will work. Hehe.
• to get to use rails_upgrade you have to have a working rails site, so I had to collect all gems in the correct versions
• using rails_upgrade plugin to check/and upgrade routes/gems/...
  • the routes generated where correct
  • the gemfile I had to edit manually (the used gems were not specified in the environment.rb using config.gem)
  • also the generated application.rb needed to be edited (we added a lot of initializer code there --> should move to an initializer!)

Then I switched (with rvm) to ruby 1.9.3 and ran bundle install. Then I created a fresh 3.2.13 project and copied:

• the scripts folder verbatim
• the config/boot.rb, config/environment.rb and config/environments/*.rb (make sure to check and keep any changes you made)
• the Rakefile (idem)

Then I was good to go!! We still have a lot of vendor/plugins I need to convert, one in particular I need to convert to an engine (gem).

Things I still need to do:

• convert vendor/plugins:
  • convert engine to gem
  • convert others to lib/plugins
• move assets to asset pipeline

And then I should have a running webapplication again. Wow.

Of course: the sites have no tests at all (my bad as well: when I started rails I did not know about testing), so will have to still add those. Starting with cucumber first, and add rspec later, when I touch the code (working from the outside in).

For my current employer I help in building a heavy javascript based website/application. Using a javascript-based site, with a lot of ajax, you have to make sure the user can still use the back button without breaking the experience.

So we used the HTML5 history, and tested that on Firefox and chrome and it worked just fine.

Of course, our first client is using IE9, and it breaks completely.

The most famous library to port HTML5 history behaviour to all browsers is history.js. Unfortunately I encountered a few very specific issues with it:

• it uses statechange event, so it is triggered when pushing or popping a new state, and I can't tell which change it is. I am only interested in the pop state. This is akward, but fixable.
• we are building a SPARQL browser, so the url's we build contain RDF identifiers, which are URI's. History.js just can't handle that. It will unescape the uri's, and thus break the built url and stored state as well. This was not simply fixable at all. It was supposedly fixed in the dev-branch, but even that did not work

So I had to go looking for an alternative, with the following characteristics:

• support the same API as the HTML5 history, or as close as possible
• allow to build urls containing escaped uri's
• and of course: work on IE9 and up

And luckily, I found that library: HTML5-History-API, which is an exact implementation of the history API.

The only change was my popstate event (and include the javascript library, of course). Before it was implemented as follows (coffeescript) :

  window.addEventListener "popstate", (e) -\> 
    state = window.history.state 
    state = event.state 
  if state 
    if state.query != undefined 
      update(state.query)

and now it looks like:

  window.addEventListener "popstate", (event) -\> 
    event = event || window.event 
    state = event.state 
    if state 
      if state.query != undefined 
        update(state.query)

And then my code just worked on IE9! Awesome :)

There are different reasons why a ActiveRecord::StaleObjectError would occur. Sometimes it is caused because rails does not have a stable identity map yet. This means that if the same object would be retrieved via different associations, your rails process would not know they point to the same object, would keep different copies of the same object and that could easily cause a StaleObjectError if you attempt to change both.

Such an occurrence needs to be fixed structurally in your code. But sometimes you can get StaleObjectError because things happen at the same time. There are two ways to handle this:

• make sure that controller actions which affect the same object, are always executed sequentially. One way to achieve this is to lock the object you want to update in the database. Then all other processes wanting to update the same object will have to wait until the lock is released (and the lock can be acquired). This is a valid approach but costly, intrusive (you need to explicitly add code), and possibly dangerous. It is normally easy to avoid, but you have to be careful for deadlocks and locks that are never released.
• when a StaleObjectError occurs, just retry the request

Now that second option seems valid, if only it would be easy to handle. Luckily there is a very easy and unobtrusive way to automatically retry all requests when a StaleObjectError occurs. Let's just create a middleware that catches the exception and retries the complete request a few times.

Create a new file lib/middleware/handle_stale_object_error.rb containing the following:

module Middleware 
  class HandleStaleObjectError 
    RETRY_LIMIT = 3 
    
    def initialize(app) 
      @app = app 
    end 

    def call(env) 
      retries = 0 
      begin 
        @app.call(env) 
      rescue ActiveRecord::StaleObjectError 
        raise if retries >= RETRY_LIMIT 
        retries += 1 
        Rails.logger.warn("HandleStaleObjectError::automatically retrying after StaleObjectError (attempt = #{retries})") 
        retry 
      end 
    end 
  end 
end

Then in your config/application.rb you need to add the following lines at the bottom to activate the middleware (this requires a restart of your rails server):

 #
 # Middleware configuration
 #
 require 'middleware/handle_stale_object_error'
 config.middleware.insert_after ActiveRecord::SessionStore, Middleware::HandleStaleObjectError

Warning : this code will retry all occurences of StaleObjectError. For some occurrences this will not help at all, so you have to use your own judgement if this middleware is something you want in your codebase. I like this approach because it is an "optimistic" approach: it only adds extra processing when a StaleObjectError occurs, and when a not-fixable StaleObjectError it still fails as it should.

Hope this helps.

In some part of my code I ended writing the following:

self.count_processed ||= 0 
self.count_processed += 1

where self is some ActiveRecord model, and count_processed is an attribute of that model (and stored in the database). What am i trying to achieve (if it is not blatantly obvious):

• if count_processed is not initialised, make it zero
• increment count_processed

Imho this code is clear and readable, but I had the feeling it could be more concise/prettier. So I asked that question on our campfire, to see if we could come up with something shorter. Very nice to be working in a team where you can just throw up questions like these and a very useful, educational discussion unfolds. In short we came up with the following solutions.

Solution 1: to_i

self.count_processed = self.count_processed.to_i + 1

Nifty! Isn't it? Use to_i because it will handle the nil correctly. But for me this looked wrong. If I would return to this code after a few weeks, months, I would wonder why I did this way, and not just wrote self.count_processed += 1. So while the code is correct, the intent of the code is not clear.

Solution 2: concise!

self.count_processed = (self.count_processed || 0) + 1 

This is very beautiful, and the intent is also very clear. If it is not initialised, use the zero, else just use the value and add 1. Awesome.

Solution 3: change the getter

An alternative solution would be to overwrite the getter, like this

def count_processed 
  self[:count_processed] ||= 0 
end

Note the notation we used: we use self[:count_processed] because this will fetch the value from the database column. If this was a normal getter, we would write @count_processed (but that does not work for an ActiveRecord model). After redefining the getter, we can just write: self.count_processed += 1 While this will work always, does it express its intent more clearly or not? Actually you no longer have to worry about the initialisation, because it is handled, and we can focus on what we really want: increment the counter. I opted for this solution.

What about you?

Which version do you prefer? Do you have any alternative suggestions?

Since rails 3.2.4 the link_to_function is effectively deprecated (again). Everything keeps on working, but when running my specs I got a horseload of deprecation warnings. I know the optimal/recommended way is to use unobtrusive javascript, but the quickest way to fix this is really easy. Just perform the following translation:

    # before 
    link_to_function icon_tag('close.png'), '$(this).parent().hide()', :title => t('actions.close'), :class => 'close' 

    # after 
    link_to icon_tag('close.png'), '#', :onclick => '$(this).parent().hide()', :title => t('actions.close'), :class => 'close' 

Dead-easy :) In the next refactoring I will remove all onclick blocks and replace them with unobtrusive javascript. But for now I got rid of the deprecation warnings :)

I had an issue where I was not sure the ActiveRecord::SessionStore was actually working (in hindsight: it worked). But to make sure, I needed to know what was stored in the session or retrieved.

All logging for the session-store is silenced, using Base.silence.

Obviously I was very interested in that logging, and did not find another to unsilence the logging but to add an initializer with the following code.

So in file config/initializers/unsilence_logging.rb write:

class ActiveRecord::Base 
  def self.silence 
    yield self 
  end 
end

This will unsilence the SessionStore logging. Your logging will look like this:

 ^[[1m^[[36mAREL (0.0ms)^[[0m ^[[1mUPDATE "sessions" SET "data" = 'BAh7DEkiFnF1aWN<<snipped to protect the inncocent>>iEi9mcC9kYXNoYm9hcmQ= ', "updated_at" = '2012-05-04 11:17:24.704491' WHERE "sessions"."id" = 33635

This at least allows us to verify that the sessions are stored and retrieved correctly. But how can we see what is stored inside the session? To be able to read or inspect what is actually stored in the session, you can use the following line:

session_data = 'BAh7DEkiFnF1aWN<<snipped to protect the inncocent>>iEi9mcC9kYXNoYm9hcmQ= ' 
Marshal.load(ActiveSupport::Base64.decode64(session_data))

And this will present your session data in a readable format.

This way I learned that a time-drift between our two servers caused a very obscure bug. I hope it can help you too.

Assume you have, like I did, a ruby gem that contains some javascript you want to test standalone. First you need to install the jasmine-gem. You have two options:

• either you use your gemspec to drive your bundler gemfile, so just add it to your developement dependencies
• I am still using jeweler, so I use a normal Gemfile, which jeweler parses to populate my gemspec with. Personally I find this much easier, and my workflow is much closer to any ruby development for me, this way

If you have a rails-project, starting with jasmine is easy, and takes three easy steps:

# first add jasmine to Gemfile, and then 
bundle install 
rails g jasmine:install 
rake jasmine 

Inside your gem or simple ruby-project it is equally simple, just type

jasmine init 
rake jasmine

Now you can need to edit the jasmine.yml to make sure it is running your tests and your code, instead of the example code. In my case I had to change only one line:

src_files: - app/assets/javascripts/\*\*/\*.js 

Happy testing :) Some interesting links to help you with jasmine:

• a very interesting slideshow
• jasmine homepage

I really like generative art, so I have been playing with processing for a while. Processing is an open source language (on top of java), that gives the possibilty to create images, animations, with added interactivity. First I created a simulation of raindrops, and because I wanted it to be easily configurable I used processing.js: processing implemented on top of javascript. So that becomes native processing in the browser. Allowing to interact with HTML and javascript objects easily. Publishing the sketch is just: give people the link.

Next-up I wanted to create a music visualisation. Interact with the music. I had the perfect piece of music in my head: "Endless Season" by Ken Ishii. Now processing.js does not interact with music. There is HTML5 audio, but it still is very experimental, and I did not find any API for processing music, reacting to and analysing music played. Processing (the java version) has an excellent library for this: Minim (actually more than one, but I ended up using that one). When the sketch was finished, I wanted to share it, convert it to a video. What were the options.

Use Processing itself

There are two ways to convert a processing sketch to a movie, from within processing itself:

• use MovieMaker: this requires quicktime and unfortunately does not work on Ubuntu
• when each frame is drawn, do saveFrame and then convert all frames to a movie afterwards. While in theory this should work, saving the images slowed down my sketch, and ultimately screwed up the sync with the audio. My frame-rate was not consistent enough. For straightforward stuff this does not matter, but I needed it to sync with the audio.
• There is a third option: GSVideo, but frankly, that seemed to damn hard for me, so I skipped that. So I needed an alternative approach. If it runs correctly on my screen, couldn't I just record it on my screen?

Use some sort of screenrecording/screencast software

To record my desktop, on ubuntu, including the sound from processing proved to have some issues:

• processing (i.e. java/JDK 6) does not use ALSA to create the sound, but address the hardware devices directly
• I do not want to record my entire desktop, but a specific part, of a specific size
• I want to share my video on vimeo, so it has to follow certain guidelines The first proved to be the hardest.

Recording the system audio out together with the video

On ubuntu, I found one approach to work very well for me:

• use gtk-recordmydesktop
• use PulseAudio mixer, it will allow to take the sound output as input to record
• and record away! :) But, unfortunately, since java does not ALSA but uses the hardware devices directly, PulseAudio was unable to capture the sounds. However, by accident I found out that if you export your processing sketch to an applet, and run the applet in the browser, it does use ALSA and can be recorded perfectly. Awesome. Part one solved.

Recording a specific part of the screen

gtk-recordmydesktop allows to specify an area of the screen to record, but somewhat akwardly.

Now, for exporting to vimeo, it had to follow certain fixed, optimal sizes. E.g. 640×480 for 4:3 SD video, 640×360 for 16:9 SD video, and 1280×720 or 1920×1080 for HD. And that is hard to do if you are trying to position the recording box manually.

But, as I found out here, when using recordmydesktop from the commandline, you can send those options along:

  recordmydesktop -x=0 -y=0 --width=320 --height=240 

So, if you open the Advanced->Misc, and look for the Extra options field, there you can fill in the same options, and when you press record gtk-recordmydesktop will show the bounding box that is recorded.

Preparing your video for uploading to vimeo

To upload your video to vimeo, you have make sure two things are correct:

• the screensize, which we discussed before
• the video format

gtk-recordmydesktop creates an Ogg Theora file, and unfortunately vimeo does not accept that format yet. Converting that to MP4 was hell, until I found Arista Transcoder. Using Arista to create an MP4 is easy (but you have to know it):

• open Arista Transcode
• create a new conversion
• select your file, by default called out.ogv, as the source
• select Sony Playstation - PSP as the device
• press create!

This will create a file called out.mp4 which is just perfect for vimeo, including the sound.

The result

Twotoned from DIXIS on Vimeo.

I was investigating ways to generate pdf's in Ruby on Rails, but I had one enormous constraint: it had to deploy on heroku.

There are two very different ways to generate pdf's in ruby:

• use prawn: it is pure ruby, very powerful. It has it's own DSL, that unleaches all the power of building a PDF, but at the same time: it seems to be very hard and tedious.
• use some sort of HTML to PDF conversion. In ruby there exists two gems: wicked_pdf and PDFKit, both use wkhtmltopdf under the covers. I dreamed of having a view magically converted to PDF.

I went for the second option. Furthermore, I choose wicked_pdf over PDFKit, because I felt the rails integration was better. It allowed me to just render a view which would automatically be downloaded as a PDF.

Setting up wicked_pdf in Rails to run on heroku

Luckily, getting it running on heroku proved to be incredibly easy: just including the correct gem with the binaries that work on heroku.

In my Gemfile I added the following:

 gem "wicked_pdf"
 gem "wkhtmltopdf-heroku", :git => 'git://github.com/camdez/wkhtmltopdf-heroku.git' 

And, then, inside a view you want to render as pdf, write something like

 respond_to do |format|
   format.js 
   format.pdf { 
     render :pdf => "show", :header => { :font_size => '8', :right => '[page] of [toPage]' }, :footer => {:font_size => '8', :right => 'Generated by jottinx.com' } 
   } 
 end

Then, you will still have to create the view, show.pdf.erb. Just make sure your view renders HTML and it will be converted to PDF correctly. That is just awesome.

Hope this helps.

The Problem

For jottinx I wrote a small piece of code that allowed to sort items using drag and drop. Of course, after writing it and making sure it works (manually), I want to make sure it keeps working. So I add a test, using cucumber.

My scenario actually looks pretty straightforward:

 @javascript
 Scenario: Dragging a book
   Given I authenticate as a "default user"
   And I add a new book with title "a-new-book"
   And I add a new book with title "another-book"
   And I add a new book with title "one-last-book"
   And I drag book "one-last-book" to the top
   Then book "one-last-book" is at the top of the list

The difficult bit was: how do I implement the dragging?.

Actually it seemed straightforward, because capybara has a method called drag_to. So I implemented the step like this:

 When /^I drag book "([^"]\*)" to the top$/ do |book_title|
    drop_place = page.find(:css, 'ul.sortable-books li:first')
    page.find(:xpath, "//a[@href='##{book_title.parameterize}']").drag_to(drop_place)
 end

But, unfortunately, this did not work. I googled around a bit and found the following two similar questions:

• drag_to with jquery sortable list
• drag_to don't work on sortable elements

The short conclusion: it does not work, and it is a combination of how jquery implemented the sortable element, and the fact that the selenium driver does not support it yet. So, refraining to using the selenium driver directly does not help either. After some more googling, I found a similar question on stackoverflow, and there I found the solution.

The solution

Enter jquery.simulate.drag-sortable.js. It is a script that will allow you to simulate dragging in a sortable object by issuing a simple javascript command:

 // drag item down one position in the list
 $('#itemToDrag').simulateDragSortable({ move: 1 });

If move parameter is negative, it will move up. And down if positive. If you include the js inside your project, you can easily test that out inside your Chrome console. It just works. Awesome piece of work. To use that in a step-definition, just write:

 When /^I drag book "([^"]\*)" to the top$/ do |book_title|
    page.execute_script %Q{ 
      $('.sortable-books li:last').simulateDragSortable({move: -4}); 
    }
 end

Hope this helps :)

This release contains

• paginate really large notebooks (unobtrusive, when reaching the end of the page, it will automatically fetch missing notes if available)
• a user can new edit her own profile (email/password) and delete her account if she so desires
• added more copy to the about/faq page What was in the previous (undocumented) releases?
• 0.0.24: entered links are clickable after saving (they were clickable in the preview only)
• 0.0.23: redesigned landing page + fixed an error in the forgot password handling

This release contains:

• tags are clickable and we also show a clickable tag-cloud
• deleting of notebooks: after deleting now jumps to the first book
• create a new book in a modal window
• there were some problems with tags undefined popping up, I hope to have fixed that If you have suggestions or encounter any problems, please let me know.

Released some small improvements:

• somehow it was impossible to save tags: that is now fixed
• links are now opened in a new tab/window

This new version contains a new layout setup. I see some room still for further improvement, but for now we get

• better use of complete screen estate
• faster loading of first page (as we start loading book after rendering the rest first)
• books only get loaded when they are needed Hope you like it. What is next:
• showing a clickable tag-cloud
• making sure the sidebar and header is always visible As always, I am happy to hear your suggestions and ideas.

Released 0.0.18 includes:

• improved Markdown editor with immediate preview
• some smaller improvements: you can now click on links immediately, flash-messages disappear, notes show the creation-date Nextup I will most likely tackle the general layout. Replace the tabs by something smaller. Make more use of the screen in general.

Release 0.0.16 includes:

• added search functionality
• after import, jump to imported book Note that you can always follow-up on the status and vote for changes on my ticket-board.

Yesterday I released 0.0.15, nothing much new, just further improved the import from Google Notebook, after I received some feedback. I also started using Trello to keep track of "things to do". Check it out here. When you check the Trello-board, you will notice I plan to work on some search functionality next, and I want to do something about the layout. Too much space is lost now.