I am developing Ruby on Rails on Windows platforms mostly. But using Rspec and cucumber on windows has a very strange side-effect: all a's are missing as can be seen from the following screenshot: Luckily, after some very extensive googling, i found a single blogpost with a fix! Apparently it has something to do with UTF-8 encoding, and the simple solution is that you need to change the encoding of the current command prompt. This can be achieved via a simple call before you start: [sourcecode] chcp 1252 [/sourcecode] The aforementioned post then proposes to adjust the cucumber.bat to not have to type this every time. This is all good for cucumber, but not for rspec, and anyhow, every new update of the cucumber i would need to apply this fix again. I was thinking that it might be possible to set the default codepage, which is 850 on my machine, to 1252 permanently. As this blogpost mentions, there are two different ways to achieve the wanted result.

Change the codepage system-wide

This can be done in the registry. [sourcecode] [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Nls\CodePage] OEMCP=1252 [/sourcecode] But one commenter notes this is not without risk.

Only for command prompt

An alternative way is to put make sure that each time a console is opened (cmd.exe) the codepage is set automatically. [sourcecode] [HKEY_LOCAL_MACHINE\Software\Microsoft\Command Processor] Autorun=chcp 1252 [/sourcecode] This will only work for console windows in which you run cmd.exe, which is just what i needed.

...or software patent troubles again

The weirdest news i read yesterday: I4i is not out to destroy Microsoft? In the article is described that some small company, i4i, has apparently won a case against Microsoft, claiming the Microsoft Word violates a patent they hold. More specifically: patent5787449. The Texas judge ruled that Microsoft has to pay a fee/fine of 200 million dollar, AND that Word versions 2003, 2007 and future versions can no longer be sold. And the problem only arises when clients or customers use a feature called "custom XML". What? Let's rewind here for a bit. What does that patent actually say? Let me summarize it for you. It describes two things: 1) it describes a system where the content is seperated from the actual representation, and 2) using a dialect of SGML. Now note: this patent was filed in 1994, before XML even existed (only since 1996), and the patent was finally issued in 1998. I tried to read the patent-document as some kind of specification, but i am not even sure what it is actually protecting. An algorithm? The idea? Because the idea of seperating content and representation is a common idiom. We all do it. Websites do it. We use css all the time. We use templates in Word ... The idea of using metacodes (like Title, Chapter, ...) and then looking up their definition the replace in the actual document (this is the example from the patent). So it must have something to do with the specific implementation: they use an XML file. Now comes the really scary part. XML is really powerful, and Word is using a set of XML files, packaged in one zipped archive as their file format since Word 2003. This is really awesome. This is more open than any binary format. Now anyone who can read can actually create or process Word-files. Microsoft calls it Open Office XML or OOXML, but that doesn't really mean the standard is really open. Only Microsoft can amend the standard, and the other big standard, OpenDocument (ODF), is the fileformat originally used in OpenOffice.org (you see the deliberate confusion in the naming?), is totally different, not supported. A missed opportunity. But that is another discussion all together. Now, i4i has no problem that Microsoft uses an XML file, just as long as the format is fixed. Meaning: you specify which tags you can read, are defined (for instance using DTD), and those tags are fixed. But Word allows you to add Custom XML, you can add your own fields into any Office document. And i4i claims that is the problem. So now i am lost. This has nothing to do with seperating content and presentation (that was what the patent was about, right?). It could have something to do with how the extra tags are added, the custom XML. If they would use a lookup-table, to identify the custom tags, then they use a technique that is described in that patent. But hey, wait, does that mean you can not use custom xml tags anymore? I have used tons and tons of xml, with no predefined DTD, so that the file format is extensible, and users can add their own extensions, and keeping the definition of the tags seperate. In my personal opinion, and i think i am not alone, Microsoft is falsely convicted. There is no technological ground this patent can hold. This all comes back to the old software patent discussion. What is patentable and what is not. It seems so unnatural to patent software, algorithms, they are the building blocks upon which we learn and build the actual stuff. And the actual stuff (software/websites) is most of the times so general on the one hand, but very specific to the customer, i don't see how these could be patented. I read that the way a website operates can be patented in the USA. Back to the patent '449. In the press release found on i4i's website, they claim that Microsoft infringes claims 14,18 and 20 from the patent. Those claims refer to the ability of using a lookup-table, being able to generate it, being able to read and replace the values, and being able to use more than one. That is so general! They could sue anyone who is using xml and some kind of look-up mechanism. I think the patent should just be declared invalid. But i guess that is easier said than done. Speaking about the seperation between content and representation, speaking of metacodes, reminded me of Donald Knuth, inventor of TeX, and author of the greatest series of books of all time: The Art of Computer Programming. So let me point you to a letter by that same man, claiming mathematical ideas or algorithms should not be patented. Final thoughts: i4i LP is a company that guards the patents from i4i. So this could be some kind of patent troll? Secondly, they choose Texas, where a large number of these kind of cases have been won, because of their technological inaptitude. I am looking forward to Microsoft's next step. What are your thoughts?

PL/SQL package: lessons learned

Working with Oracle Spatial a lot, I found a great source of information in the Spatial DB Advisor, which besides a lot of interesting articles, also offers his source-code and PL/SQL packages for free. The site is not very user-friendly (trouble of seeing the threes through the wood), as finding the free packages is something that i only succeed in coming from Google :) A shortcut: you can find them here. But a word of warning: if you try to install this package in an existing schema, it will delete all indexes and tables. This is something i had to discover the hard way (painful i might add). So i have adapted my version of create_test_data.sql, where at the top of the file all indexes and tables of the user are dropped. Nevertheless, extremely useful package. I will list my favourites (for now):

• isCompound: checks whether a geometry contains any circular/arc elements
• ConvertGeometry: converts any special elements - circulararcs, rectangles, circles - in a geometry to vertex to vertex linestrings
• to_2d: converts a sdo_geometry to 2d. Very useful for us, as geoserver can't draw 2d, and we only use 2,5d anyway. Meaning that we always use a topview, and the z is the elevation level of the ground-level. Or for pipelines: below ground :)
• sdo_mbr: function to determine the minimum bounding rectangle! i will use this to adapt my script to fill user_sdo_geom_metadata. Cool :)
• functions to investigate the data of sdo_geometry (number of rings, number of vertexes, number of coordinates, the coordinates itself, ...)
• functions to investigate the meta-data
• functions to manipulate your sdo-geometry: scale, rotate, affine transformations, move, adding and removing points, ...
• ... and lots more ... In short: awesome!! So after getting a terrifying scare, now rebuilding the databases (and adapted the script: won't happen to me again). I mailed the original creator to add a note of warning on his site too. Might help other absent-minded developers :) I also had another smaller problem, because the install-script overruled my ORACLE_HOME and PATH definition, which made sure SQL*PLUS was not found. But that was easily cured. I am very grateful that he puts up this package for free. And i learned another great lesson today: not to get too enthusiastic before i am sure that it all works ;)

I have been trying the Windows 7 beta, and i must admit it is pretty awesome. What is less awesome is the fact that now the RC is released, and i have to reinstall all my Windows 7 machines (three --i admit i was a bit too enthusiastic). And if i do that, once the final version is released, i have to re-install them again. For my standard day to day use not too bad, but not for my development machine. So i am back to Windows XP now. With a great new look: Zune XP Theme. Nice :) Working with a lot of open source products lately, so it would be nice to use an open source windows alternative. For the moment it would still be very hard for me to just not work on Windows. I still have to develop for .NET and MicroStation and AutoCAD, all only working on Windows platform. So I hope that ReactOSwill become more mature soon. It is really improving fast lately, but still not completely ready to replace Windows. I will have a test-drive real soon. What are the other options? Run Linux and do my Windows work inside a VM? Buy a Mac and do the same? Any suggestions?

When storing spatial data into Oracle, there are a few steps one needs to complete (as shown here too):

• insert the data (obviously)
• update the USER_SDO_GEOM_METADATA table. This table specifies for each column the bounding box and the SRID (coordinate system).
• create a spatial index I have created a script to this automatically for me, once all tables are filled with their data (e.g. after import or after using FME to import your data). [sourcecode language="sql"] set serverout on DECLARE schema_orig varchar2(100) := upper('&user_orig'); CURSOR ctab is select TABLE_NAME from sys.dba_tables where owner = schema_orig; tn sys.dba_tab_columns.TABLE_NAME%TYPE ; CURSOR ctabcol is select column_name,DATA_TYPE from sys.dba_tab_columns where owner = schema_orig and table_name = tn ; collist varchar2(2000) ; query varchar2(2000) ; column_name varchar2(200); has_geometric_column boolean; col_count number; BEGIN -- dbms_output.enable(1000000); FOR ctabrec IN ctab LOOP tn := ctabrec.table_name ; has_geometric_column := false; col_count := 0; for ctabcolrec IN ctabcol LOOP if ctabcolrec.data_type = 'SDO_GEOMETRY' then has_geometric_column := true; col_count := col_count + 1; column_name := ctabcolrec.column_name; -- insert data into user_sdo_geom query := 'INSERT INTO USER_SDO_GEOM_METADATA (TABLE_NAME, COLUMN_NAME, DIMINFO, SRID) VALUES ('''||tn|| ''' , '''|| COLUMN_NAME ||''', MDSYS.SDO_DIM_ARRAY( MDSYS.SDO_DIM_ELEMENT(''X'', 0, 10000000, 0.000005),MDSYS.SDO_DIM_ELEMENT(''Y'', 0, 10000000, 0.000005), MDSYS.SDO_DIM_ELEMENT(''Z'', 0,10000000, 0.000005)), &srid)'; dbms_output.put_line(query); BEGIN execute immediate query; EXCEPTION WHEN OTHERS THEN dbms_output.put_line('**** failed to insert row into user_sdo_geom_metadata for '||tn||'('||column_name||')'); --rollback; ignore any errors! there will be existing columns END; -- create spatial index query := 'CREATE INDEX '||tn||'_'||to_char(col_count)||'_SX ON '||tn||'('|| COLUMN_NAME ||') INDEXTYPE IS MDSYS.SPATIAL_INDEX'; dbms_output.put_line(query); BEGIN execute immediate query; EXCEPTION WHEN OTHERS THEN dbms_output.put_line('**** failed to create spatial index for '||tn||'('||column_name||')'); --rollback; ignore any errors END; end if; END LOOP; END LOOP; commit; END; / [/sourcecode] The script has two parameters: the schema from where to scan all tables, and the srid (coordinate system) which needs to be filled in. Any thoughts on this? suggestions?

I am now working in Ruby on Rails for about a month. I have done a few smaller things before, investigating, trying out; but now I am really building a full application site in Rails! It feels great :) But of course, just starting out in such a new environment, both ruby and rails, I encountered a lot of initial problems. One of them is showing international, accented characters, like é, è, ê ... from the database. Nor Rails nor irb could show the correct characters. I really needed to solve this problem, so this started an investigative journey.

The symptoms

When displaying a table of results from Rails in any browser, all accented characters were shown as white question marks inside a black rhombus (diamond?). When i tested this with irb i also got very weird results. When using TOAD the results of a query are displayed correctly. But maybe TOAD just does it very clever? If i use the GUI version of SQL*Plus (sqlplusw) the results are also shown correctly. But when i use the console (command-line) version of sqlplus, the characters were also distorted. So maybe the clue could be found there.

Oracle NLS-LANG

No matter what character-set the Oracle database uses to store the data (e.g. UTF-8, UTF-16) the data is always converted to the clients character set. This is truly an amazing feature. This depends on a setting called NLS_LANG. Not sure wat it quite stands for. NLS_LANG is built up as follows:NLS_LANG=language_territory.charsetThe NLS_LANG property is set by default in the registry to the correct Windows codepage.NLS_LANG="AMERICAN_AMERICA.WE8MSWIN1252"But why doesn't it work inside the command line console? Apparently because it used a different codepage. If you type chcp (requesting the codepage) at the DOS prompt, it would normally return 437 (it did on my machine). So you would need to enter`

set NLS_LANG=american_america.US8PC437 require 'oci8' OCI8.new('user','pw','db').exec('select * from emp') do |r| puts r.join('|'); end `in your irb and then all results would be displayed correctly. The crucial line being the correct setting of NLS_LANG. Wow! I got my results correctly in ruby! Now i was in the assumption that Rails would be a piece of cake, but that was wrong.

Fixing Rails

The easy idea would be to set NLS_LANG in Rails correct, before the oci8-library is required. My first approach was to set the NLS_LANG in the first line of the environment.rb with the following line:ENV["NLS_LANG"] = "AMERICAN_AMERICA.WE8MSWIN1252"But this didn't work. I am using NetBeans 6.5, and it took me a while to realise that if I edited a file to contain special characters (fixed text, e.g. on a menu) it would work. NetBeans (or Rails for that matter) standard works with UTF-8. So all files are encoded in that way. The easy solution would be to useENV["NLS_LANG"] = "AMERICAN_AMERICA.AL32UTF8"But that didn't work. I tried the alternative AMERICAN_AMERICA.UTF8 but that didn't work either. I am just guessing here, but i think somehow the NLS_LANG setting didn't get picked up in the Rails environment. It kept using the registry setting. So I tried to turn it around: make Rails use the windows codepage instead of utf-8. This took several steps:

• change the default setting inside NetBeans to use the correct code-page instead of UTF-8. This in fact only affects the format of the files that are saved, but it is nevertheless important that all files being served are in the same format. Also convert all previously edited files from UTF-8 to standard ASCII. I used an editor to do that.

• added the following to application_controller.rb:` before_filter :headers_iso

  def headers_iso

  make sure the charset matches the default Oracle NLS setting

  headers["content-type"]= "text/html; charset=windows-1252" end `

• added the correct META-tag to application.rhtml in the HEAD-section:``Note: this meta-tag is not really needed, it has no real effect. And that finally worked! :)

I have this machine at home, running XP, and all of a sudden it is getting incredibly slow at startup. I installed Ad-Aware to make sure i don't have any hidden spamware running. It found nothing. So then i tried to find ways to make XP run or boot faster again. I know that the really failsafe way is re-installing my XP but i don't really have the guts to do that, reinstall all drivers and programs. So i tried my friend, Google. I found several articles allowing me to tune my xp-performance and startup time, i applied the ones i found sensible, the ones that were mentioned accross different pages. In short i think the best things you can do is uninstall unneeded programs, make sure that unwanted programs in the background can't start (msconfig). Bootvis seems interesting too, not entirely sure if it helped a lot. Defrag of course! :) Making sure your pagefile can't get defragged by making it large enough, preferable on a separate disk from your boot-drive. These changes seemed to improve my boot, although it is no where near a quick 20 seconds now :) It still takes a few minutes to get my computer completely ready. But suddenly i also lost my network connection. I was puzzled, did i stop one service too many? :) Everything seemed to work fine, but my network-card did not look for an IP address (DHCP). I tried to start my Firewall and i got an error stating the WSAStartup had failed to start, or was not started. WSAStartup? What is that? Looking on the internet i found a small tool, fixing WinSock problems, possibly related to de-installing programs. You never know. And i had no other clues to follow anymore. The tool is called WinsockXPFix. It supposedly could also backup my registry, but that failed in my case. So i just started it, in blind faith, knowing i had to re-install anyway if it wouldn't work. But guess what: i did! The program ran, rebooted automatically, and everything was back in working order! Windows XP, at the time, was promoted for its boot performance, it's self-tuning capabilities, yet over time it only becomes slower and slower. How does Vista handles this? Or Windows 7? Or is it inherent to Windows OS, and should we look at other alternatives. Sometimes i hate how much is done behind the scenes, how much is added. Each vendor or software installs its own version of "Automatic Updates", slowing down my system. Should I look at Mac OS X? Or Linux? Yet Linux still feels to "hardcore" for me, but maybe i am wrong. Should we look at different open source alternatives (in order of viability):

• Visopsys? this seems like a single developer's playground. Not quite sure what kind of future this holds. But what a great effort to complete alone, though :)
• Syllable? not quite sure about this one either, either a Linux-based server, or a desktop which is based on Amiga :) That is very nice, i sometimes long for the old times with all the different operating systems out there: QDOS (Sinclair QL), AmigaDOS, Acorn Archimedes, Next. But reviving something like that will never be able to compete with anything commercial. Although i know Mac OS X is heavily based on Next, and rightly so. Next was anyway far ahead of its time. Not to mention they looked nice. I have a soft spot for matte black :)
• SkyOs: this seems promising, seems to be based on BEOS, which was very promising at the time. It has a lot of applications (open source) available, and a concept of a Software Store: a central place to find all software. A bit like AppStore maybe, and sounds like a good idea.
• Linux, it all its different flavors
• Wine allowing to run Windows-application under Linux, makes Linux even more interesting :)
• Hackintosh: running Max OS X on cheap intel computers. Sounds like a viable option too, albeit a little illegal, since Mac OS X is only allowed to run on Apple hardware. That is a shame though. Would be tempted to try that, just to know for sure Mac OS X is my thing before i buy such a machine.
• ReactOS: i have no idea how good this is at the moment, but this sounds like an extremely interesting idea to me. Create an open source, improved, version of Windows XP. You can run all current Windows applications on it. I guess they hope to stay up to date. Looks like a very interesting option. So i am also not quite sure what the impact should be, changing OS. I guess application-wise it doesn't matter too much nowadays. There is Open Office, AbiWord, .. which allow you to most of the day to day jobs. Not sure if i could miss PhotoShop though. There is GIMP, and GIMPShop should make switching a lot easier. And such a new OS could revive my older hardware. Maybe.