When it is a blue arrow on white background, the shift-key lasts for 1 letter. But when it is a white arrow on blue background, then the shift is turned on permanently.
To activate the 'sticky' shift-key, either double tap on one Shift key, or press on the two shift-keys simultaneously. Personally, do I prefer the latter one to activate it, as it always works.
Edit: That is a very good point jacksonian. Thank you for pointing out, that it has to be activated first.