It's not clear where the word came from. Here is some etymological speculation from http://www.answers.com/topic/nonce-slang:
It is possible that it is an abbreviation of nonsense or nonsense case, perhaps
derived as British prisoner slang for prisoners who have committed a "nonsense
crime", as opposed to a crime such as theft. 
It is also possible that it comes from the acronym used in prisons to describe said
individuals: "Not On Normal Communal Exercise" or "Not on Normal Courtyard
There is also evidence for a possible connection with nancy, a derogatory term referring to effeminate or homosexual males, and with a dialectical use of "nonse" to refer to a worthless person. 
Jeffrey Archer in A Prison Diary said that "Nonce" was an acronym for "Not of Normal Criminal Element".
Interestingly, the word also has a standard meaning - 'presently', as in, "Her tendency to discover a touch of sadness had for the nonce disappeared” (Theodore Dreiser).