But I can understand - I was watching 'six days seven nights' (or t'other way round?) on the telly recently (Why? I don't know, but it was ok). Anyway, Temuera Morrison (fab NZ actor) was in it for a short time - he didn't have much to say, and I thought at first he was Australian. So, it happens to the best of us :)
The prime difference that we .au dwellers of our friends in .nz is that, per Wades post, the NZ accent tends to 'flatten' the vowel sounds - so nearly every short vowel sound is somewhere between 'uh' and 'eh'. Hence, the stereotyped taunts of 'suxty sux', 'fush and chups' etc heard in the broaded NZ accent.
But yeah, differences between .au and .nz perhaps not as distinctive as those between a Texan and a Noo Yorker, or a resident from Devon and one from Manchester.