The Benevolent Machine

N Tips About Coding for the iPhone That I Wish I Knew Last Week

2011-06-01T16:16:00.000-07:00

After my first week diving into iPhone / Objective C / XCode / UIKit development, here are some things I wish I knew when I started.

Interface Editor windows do not get saved by the "Save All Files" dialog when you launch a build. Get in the habit of command-S saving when you make changes in the interface editor or you will wonder why your UI changes are not working.
To associate a UI object (like a button) in the Interface Editor with a member variable, add instance variables which include the IBOutlet and IBAction declarations, and use a right-button mouse drag to associate the object and the field defined in the owning class. The right-button action is hard to discover by inspection!
Open the gdb window when debugging. It shows the NSLog output, and if you are familiar with gdb's command line, it's a lot quicker to use than the GUI. Commands like step, continue, where, up, down, print are super useful.
In gdb use po to print objects:
po myObject
You can print the reference count for an object in gdb like this:
print (int)[myObject retainCount]
When you use @property to declare a property for a class, XCode 3.2 will automatically create the instance variable if you don't do so explicitly - it doesn't add anything to you source file, but it is defined. Apparently this is a recent feature, and so you will see some code examples that declare the variable, some that do not - they are equivalent.
Some general rules with retain/copy/assign: use assign for delegates, use copy for NSStrings or anything else that is mutable, and use retain for most other NSObject types. Be careful about mutual retain relationships: while A retains B and B retains A, neither A nor B can be freed.
The following are NOT all the same:
    member=value
    self.member=value
    [self setMember: value]
It appears that the first version will not call the setter function, the second two will. This distinction is critically important if you are using retain setters to increment the reference count on the assigned object.
When writing classes that call delegates (which seems to happen a lot), it is smart to always pass the sending object as the first parameter of all callbacks, and in the protocol definition specify the callback with a full type specification ie (MyWorker *), not (id). This pattern ensures that when you write a class that acts as a delegate for many other classes, the delegate functions all have unique function signatures.
The static analyzer assumes method with names starting with new (eg "newPost") always return a reference-counted object that must be released by the caller. If you violate this naming convention the static analyzer will give you spurious leak warnings like this: "Potential leak of an object allocated on line ..."

Using RubyGems Gem.use_paths

2010-11-17T16:27:00.000-08:00

Here's a quick tip on Gem.use_paths that might not be obvious from the documentation.

When you initially fire up rubygems, you will generally have two directories included in your Gem path; the system gems directory, and your user gems directory if you have one. Under OS X I actually see three:

$ uname -a
Darwin euramoo 10.4.1 Darwin Kernel Version 10.4.1: Wed May 12 21:36:54 PDT 2010; root:xnu-1504.7.50~2/RELEASE_I386 i386

$ irb -r rubygems
>> Gem.path
=> ["/Users/guy/.gem/ruby/1.8", "/Library/Ruby/Gems/1.8", "/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/gems/1.8"]

Now suppose you want to add a new gem directory using Gem.use_paths. You might try something like this:

>> Gem.use_paths(nil, ["/my/special/path"])
=> nil
>> Gem.path
=> ["/my/special/path", "/Library/Ruby/Gems/1.8"]

Passing nil as the first parameter indicates you want to preserve the existing gem home directory. You can see that this command has dropped two of the original three default paths. I didn't expect that. One of those three was the home directory and was preserved, the other two were dropped.

The solution is easy; if you want to add new paths without dropping any of the existing ones, pass Gem.path plus your new paths as the second parameter.

>> Gem.use_paths(nil, Gem.path << "/my/special/path")
=> ["/Users/guy/.gem/ruby/1.8", "/Library/Ruby/Gems/1.8", "/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/gems/1.8", "/my/special/path"]

Rubygems is smart enough not to duplicate the paths - here we have passed the home directory in as part of Gem.path and it does not show up twice in the resulting path list.

Two more quick observations:

Gem.dir returns the gem home directory; that's the only one of the default gem directories that will be preserved when you call Gem.use_paths with nil as the first parameter.

Gem.use_paths does not necessarily return the new path list. It might, or it might return nil. Within rubygems.rb, the return value is the result of @gem_path.uniq!, and the semantics of uniq! are such that it returns nil if no duplicates were removed. Better ignore the return value and use Gem.path to get the actual directory list.

Errors installing the curb gem on MacOS

2010-10-28T17:29:00.000-07:00

I've run into this issue several times now, so this post is mostly a note to me of the future. Under MacOS 10.4 the following error occurs if you try to install the curb gem in the usual way.

$ gem install --user-install curb
Building native extensions.  This could take a while...
ERROR:  Error installing curb:
        ERROR: Failed to build gem native extension.

/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby extconf.rb
"-arch i386 -arch x86_64 -g -Os -pipe -fno-common -DENABLE_DTRACE  -fno-common  -pipe -fno-common $(cflags)"
checking for curl-config... yes
checking for curlinfo_redirect_time... no
checking for curlinfo_response_code... no
checking for curlinfo_filetime... no
checking for curlinfo_redirect_count... no
checking for curlinfo_os_errno... no
checking for curlinfo_num_connects... no
checking for curlinfo_ftp_entry_path... no
checking for curl_version_ssl... no
checking for curl_version_libz... no
checking for curl_version_ntlm... no
checking for curl_version_gssnegotiate... no
checking for curl_version_debug... no
checking for curl_version_asynchdns... no
checking for curl_version_spnego... no
checking for curl_version_largefile... no
checking for curl_version_idn... no
checking for curl_version_sspi... no
checking for curl_version_conv... no
checking for curlproxy_http... no
checking for curlproxy_socks4... no
checking for curlproxy_socks5... no
checking for curlauth_basic... no
checking for curlauth_digest... no
checking for curlauth_gssnegotiate... no
checking for curlauth_ntlm... no
checking for curlauth_anysafe... no
checking for curlauth_any... no
checking for curle_tftp_notfound... no
checking for curle_tftp_perm... no
checking for curle_tftp_diskfull... no
checking for curle_tftp_illegal... no
checking for curle_tftp_unknownid... no
checking for curle_tftp_exists... no
checking for curle_tftp_nosuchuser... no
checking for curle_send_fail_rewind... no
checking for curle_ssl_engine_initfailed... no
checking for curle_login_denied... no
checking for curlmopt_maxconnects... no
checking for curle_conv_failed... no
checking for curle_conv_reqd... no
checking for curle_ssl_cacert_badfile... no
checking for curle_remote_file_not_found... no
checking for curle_ssh... no
checking for curle_ssl_shutdown_failed... no
checking for curle_again... no
checking for curle_ssl_crl_badfile... no
checking for curle_ssl_issuer_error... no
checking for curlopt_username... no
checking for curlopt_password... no
checking for curlinfo_primary_ip... no
checking for curlauth_digest_ie... no
checking for curlftpmethod_multicwd... no
checking for curlftpmethod_nocwd... no
checking for curlftpmethod_singlecwd... no
checking for curlm_bad_socket... no
checking for curlm_unknown_option... no
checking for curl_multi_timeout()... no
checking for curl_multi_fdset()... no
checking for curl_multi_perform()... no
checking for Ruby 1.9 Hash... no
checking for Ruby 1.9 st.h... yes
checking for curl_easy_escape... no
creating curb_config.h
creating Makefile

make
gcc -I. -I. -I/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/universal-darwin10.0 -I. -DRUBY_EXTCONF_H=\"curb_config.h\"  -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE   -fno-common -arch i386 -arch x86_64 -g -Os -pipe -fno-common -DENABLE_DTRACE  -fno-common  -pipe -fno-common  -I/opt/local/include -g -Wall  -c curb.c
In file included from /opt/local/include/curl/curl.h:35,
                 from curb.h:12,
                 from curb.c:8:
/opt/local/include/curl/curlrules.h:143: error: size of array '__curl_rule_01__' is negative
/opt/local/include/curl/curlrules.h:153: error: size of array '__curl_rule_02__' is negative
lipo: can't open input file: /var/tmp//ccRdCdoj.out (No such file or directory)
make: *** [curb.o] Error 1

There are two solutions offered in the curb FAQ. I'm not a fan of the first solution listed; it installs and builds a bunch of ports, and after 10 minutes I begin to worry what new version issues I'm going to bring down upon myself...

sudo port install curl +universal  # don't use this one

The second solution from the FAQ is the droid you were looking for. The form listed in the FAQ will do a root install of the gem:

sudo env ARCHFLAGS="-arch x86_64" gem install curb  # use this one

The same approach also works for a user-install:

ARCHFLAGS="-arch x86_64" gem install --user-install curb

Eliminating DeltaCopy's "File name too long" Errors

2010-10-18T17:43:00.000-07:00

I use the Windows rsync server DeltaCopy to backup files from a Windows 7 box using a nightly cron job running on a FreeBSD server. DeltaCopy is a wrapper that runs the Cygwin rsync binaries as a Windows Service. This all works well out of the box with one exception: the version of Cygwin bundled with DeltaCopy doesn't support long file paths, so I get a bunch of failed copies in my nightly backup logs, like this:

rsync: readlink_stat("Profiles/07b6fq5u.default/Mail/Local Folders/Eudora Mail.sbd/Read.mozmsgs/!~!UENERkVCMDkAAQACAAAAAAAAAAAAAAAAABgAAAAAAAAA9XtW%2Fsf8BUe37SpujdFeE8KAAAAQAAAABb%2BHEGlU%2FkaQ5j0m1KGNZgEAAAAA%40earthlink.net.wdseml" (in mail)) failed: File name too long (91)

The solution is to upgrade the Cygwin libraries that are used by DeltaCopy. The current versions (1.7.7) solve this problem and are compatible with DeltaCopy. You also need to add a couple of lines to the DeltaCopy configuration file.

There are quite a few posts on this subject on line offering solutions for this, but I did not find any that contained all of the correct information in one place, so here's what worked for me (Windows 7, DeltaCopy 1.3, Cygwin 1.7.7).

Download the Cygwin installer (setup.exe) available from the Cygwin website
Install the base packages, plus Net/openssh and Net/rsync
Stop the DeltaCopy server (either from the Control Panel or using DeltaS)
Copy the files below from the Cygwin installation path (eg C:\Cygwin\bin) to your DeltaCopy installation path (eg C:\DeltaCopy).
chmod.exe
cygcrypto-0.9.8.dll
cyggcc_s-1.dll
cygiconv-2.dll
cygintl-8.dll
cygminires.dll
cygpopt-0.dll
cygwin1.dll
cygz.dll
rsync.exe
ssh.exe
Edit C:\DeltaCopy\deltacd.conf and add the following two lines at the beginning of the file.
uid = 0
gid = 0
Restart the DeltaCopy service

Git mojo - The current branch is not tracking anything.

2010-08-29T18:35:00.000-07:00

If you use git and github, you may have run into this little nasty surprise when creating a branch locally and pushing it to the remote repository:

# create a local branch
$ git branch mybranch

# push the branch to the remote repository
$ git push origin mybranch

# modify some files and commit the changes...
$ git commit

# push the changes to github
$ git push
fatal: The current branch mybranch is not tracking anything.

The problem here is that when the branch was created and pushed, it did not automatically set up the configuration necessary to link it to the remote repository. Specifically what is missing in .git/config is something like this:

[branch "mybranch"]
remote = origin
merge = refs/heads/mybranch

There are a few routes to get you back to good:

You can edit .git/config in your local working tree and add the missing lines. This smacks of effort.
You can use the following commands to add the missing configuration lines to the config file:

$ git config branch.mybranch.remote origin
$ git config branch.mybranch.merge refs/heads/mybranch
You can use git's --track (or -t) option in the original branch command to avoid this problem in the first place.

Now if only I could remember --track each time I create a new branch.

Replication Lag - What Is It and How is it Cured?

2010-07-18T23:31:00.000-07:00

So your database is becoming a bottleneck, and you are thinking about moving to a distributed database? Yeah, me too. The replication support in MySQL is easy to set up, solid as a rock, and seems to be bandwidth efficient. Problem solved! Well maybe not as easily as that. Here's what I've learned migrating a complex system to a replicated environment.

The obvious challenge with replication is that all writes must be directed to the master. If you are thinking about replication, you probably already have a solution in mind for that. In frameworks like Ruby on Rails it turns out to be relatively easy to intercept ActiveRecord writes and route them to the master. That's a good first step, but have you considered the ramifications of replication lag on your system?

What is Replication Lag?

With a typical MySQL database running on modern hardware, with a handful of replicas, it may take a second or two for a change written to the master to be visible in a read from a slave. Those few seconds of lag can cause major headaches.

Replication lag plays havoc with the common read-update-read cycle. Consider a hypothetical example of a user logging into a site with session cookies and a session record stored in the database.

The login request creates a new session, and returns a page that sets the session cookie and an HTML page which in turn will load some additional assets - images, scripts, etc.
A moment later the asset requests arrive with a cookie referencing a session record.
The session record query is read-only, so it is directed to a replica. The session record has been written to the master, but has not yet propagated to the replica, so the query fails. This looks to the system like an invalid or expired session. The request fails.

This scenario applies to any real-time interactions where the user is modifying records and reloading a page which should reflect those results. Here are some strategies that might help. We ended up using all of these in various combinations, depending on the usage of the individual database entities.

Read From Master For Time-Critical Entities

If you can identify a few key database objects that are time-critical, you can force those to be always read from the master. This obviously defeats the benefit of replication, but if you only have a small number of time-critical objects, that may be all you need. Lucky you.

Fail-over to Master

It became evident while looking at replication issues on our system that whenever a database query specifies an explicit value for the primary key, the caller has a sound reason to believe that record exists. Thus if that query fails with record-no-found, we can reasonably suppose that the record hasn't been propagated yet, and so we can retry the same query on the master database. This turns out to be pretty easy to implement at a low level in ActiveRecord. This solves the replication lag issue cleanly for any table that is getting periodic inserts, but the records are either never updated or the updates are not time-critical.

Caching Critical Query Results

If your database access is nicely abstracted (maybe through ActiveRecord) the above techniques can be implemented without even touching your application logic, which is very reassuring. Sadly there are still some corner cases left unsolved. For example you may need to find all records related to the current session record, and you don't want to miss a record that was just inserted.

If you can't afford to load the master with the query, you can use a tool like memcached to maintain an in-memory index of the associated objects. A short timeout (a few minutes) on the cache is fine because you really only care about the situation where the data has just changed. Recognize that you aren't so much caching the records, but you are caching sets of record references that would otherwise be returned by a query. Remember that memcache has a 1Mb limit to each cache entry, so design your caching strategy to ensure you don't hit this limit.

Image Size Problems with JQuery and IE

2010-06-15T21:43:00.000-07:00

I've been working on an image preloader for a JavaScript gallery / slideshow. I use a spinner image in place of the actual images until the images finish loading, and replace the temporary image with the actual image in the jQuery load event handler.

actualImg = jQuery('<img />');
actualImg.load(
  function(){
    tmpImg.replaceWith(actualImg);
    fitImage(actualImg);
  }
).attr('src',imageUrl);

This approach works well in FireFox and Chrome, but under IE8 I found that the fitImage call failed because the image width and height were both undefined.

Why doesn't IE8 set the width and size prior to the jQuery load callback?

It turns out that IE8 will only set the width and height if the img node is already inserted into the DOM. I ended up with something like this (change in the first line only).

actualImg = jQuery('<img />').appendTo(canvas);
actualImg.load(
  function(){
    tmpImg.replaceWith(actualImg);
    fitImage(actualImg);
  }
).attr('src',imageUrl);

Getting Descriptive Error Message from the Solr Ruby API

2010-04-02T02:39:00.000-07:00

I've been working on calling Solr (the search engine) using the Solr ruby gem. If the Solr server throws an error, the Ruby API call returns a pretty unhelpful error:

Net::HTTPFatalError: 500 "Internal Server Error"

I can look the error up in the Tomcat logs, but it would be a lot nicer if I could get a meaningful error message through the client API.

The good news is that I found I can retrieve the entire HTTP response object via the Net::HTTPFatalError's 'data' attribute. The HTTP response contains the full text of the error returned by Solr. The bad news is that Tomcat, by default, formats the error using the org.apache.catalina.valves.ErrorReportValve class. The output is human-readable, but not particularly machine-readable.

So the upshot of all of this is that I can trap the exceptions returned by Solr::Connection#query and extract the Solr error text like this:

    begin
      @solr.query(query, modifiers)
    rescue Net::HTTPFatalError => e
      if e.data.kind_of?(Net::HTTPInternalServerError)
        if /<h1>([^\n]+)\n/ =~ e.data.body
          puts "---- Solr exception -----"
          puts $1
        end
      end
      raise
    end

Which yields something like this:

---- Solr exception -----
HTTP Status 500 - there are more terms than documents in field "display_name", but it's impossible to sort on tokenized fields

Despite being cryptic and HTMLified, that message turns out to be extremely useful.

Zappos HTTP Humour

2010-03-30T18:46:00.000-07:00

I've got to love a company that puts humour in their HTTP headers where it will be seen by almost no one. I saw it!

HTTP/1.1 200 OK
Server: nginx/0.8.31
Content-Type: text/html;charset=UTF-8
Content-Language: en-US
X-Core-Value: 10. Be Humble
X-Recruiting: If you're reading this, maybe you should be working at Zappos instead. Check out jobs.zappos.com
Cache-Control: max-age=1998
Date: Tue, 30 Mar 2010 10:18:13 GMT
Transfer-Encoding: chunked
Connection: keep-alive
Connection: Transfer-Encoding

Writing a Custom Zabbix Media Script for Google Talk

2010-03-30T18:12:00.001-07:00

Zabbix includes out-of-the box support for sending alerts by jabber, which is great. I've never been able to get it to work on the Google Talk network, and it from what I've read, others have the same problem. In Zabbix terminology, 'media' are notification methods, like email, twitter and jabber. Fortunately Zabbix lets you add custom media via scripts, so we can replace the built-in jabber with one that works with gtalk.

I found the Zabbix documentation on this a bit thin on this subject. The key things you need to know to write a custom media script are:

The script must be stored in the script path you specified in the configuration file:
AlertScriptsPath=/home/zabbix/zabbix_server/scripts/
You cannot, it seems, provide a fully-qualified path to an arbitrary file location.
The arguments passed to the script are, in order:
- recipient - ie the email address or phone number
- subject - text defined by the triggered action
- message - text defined by the triggered action

I'm using ruby for the jabber script, with the xmpp4r gem. The script needs to be modified to insert the username and password of a Google Talk account you create for zabbix. You will also need to add the zabbix user to your contact list so it has permission to contact you.

#!/usr/bin/ruby
require 'rubygems'
require 'xmpp4r'

to = ARGV[0]
subject = ARGV[1] # not used
message = ARGV[2]

username = 'zabbix@somedomain.com'
password = 'somepassword'

host = 'talk.google.com'
resource = 'Zabbix'
port = 5222

#Jabber::debug = true # Uncomment this for verbose logging
jid = Jabber::JID::new("#{username}/#{resource}")

cl = Jabber::Client::new(jid)
cl.connect(host, port)
cl.auth(password)

# Create a message
m = Jabber::Message::new(to, message)

# Send it
cl.send m

You still need to configure this new medium in Zabbix, configure the medium for each user, and add this medium for notification to the appropriate actions.

"Operation not permitted" when installing Ruby gems

2010-03-18T18:41:00.000-07:00

I love the ruby gem system. I recently built a runtime dependency-following gem installer, and found the gem system incredibly versatile and extensible. One tiny complaint is that it barfs installing user-local gems if they include native code. The installer tries to chown the file, and the install fails with an error like this:

install: /home/guy/.gem/ruby/1.8/gems/zipruby-0.3.6/lib/zipruby.so: chown/chgrp: Operation not permitted
*** Error code 71

Installing as root works fine, but sometimes I do not want gems to have global visibility. The solution is easy, but not well documented. If the environment variable RB_USER_INSTALL is set, the installer will skip the chown step. In csh, you can do this:

> setenv RB_USER_INSTALL
> gem install zipruby

If you are invoking the installer from ruby code, you can set the environment variable like this:

ENV['RB_USER_INSTALL'] = 'true'
Gem::GemRunner.new.run(args)

Nokogiri, XPath and UTF-8

2010-03-17T21:28:00.000-07:00

Here's an interesting little feature of the Nokogiri HTML parser for Ruby that caused me some pain. It turns out that when you retrieve a string from a Nokogiri document using the xpath functions, the result is converted to UTF-8, regardless of the encoding of the source document. If you do not realise this and use iconv to re-map it to UTF-8, you will get garbage.

Consider this test:

def get_html(charset, code)
html = "
<html>
<head>
<meta http-equiv=\"content-type\" content=\"text/html; charset=#{charset}\">
</head>
<body>
<div id=\"1\">
 Test #{code};
</div>
</body>
</html>
"
dom = Nokogiri::HTML.parse(html)
end

# extract from a utf-8 html page
code = "&#xf6;"
charset = 'utf-8'
dom = get_html(charset, code)
PP.pp dom.search('//div').first.text

# extract from a cp1122 html page
code = "\x6a"
charset = 'CP1122'
html = get_html(charset, code)
PP.pp dom.search('//div').first.text

I found that both of the tests produce the same output:

"\n Test \303\266;\n "

According to the letter database, \303\266 is the correct octal representation for this character in utf-8.

Adding groups of items to the Zabbix Agent

2010-03-16T18:56:00.000-07:00

It can be a bit tedious adding custom items to Zabbix. Until now I've been adding separate entries to the agent configuration file like this:

UserParameter=custom.tracker.backlog,/home/zabbix/bin/custom_plugin tracker.backlog
UserParameter=custom.item.count,/home/zabbix/bin/custom_plugin item.count
UserParameter=custom.event.count,/home/zabbix/bin/custom_plugin event.count

It turns out that Zabbix has a configuration syntax that greatly simplifies this. As an added bonus, you don't have to restart the zabbix agent when you add new items.

Replace the above lines in zabbix_agentd.conf with a line like this:

UserParameter=custom[*],/home/zabbix/bin/custom_plugin $1

Then, in the Zabbix item configuration, identify each of the items supported by your script. In my case they are named custom[item.count], custom[event.count] and custom[tracker_backlog].