Better endianness handling

[tkittel]

Currently we assume that machines can be classified strictly as big or little endian, and simply prevent files from being opened on a platform with a different endianness than where it was created.

Ideally, we should instead change our i/o code to be insensitive to platform endianness. However, one issue is if people put numbers inside binary blobs. In that case, we might have to provide convenience functions for users wishing to make numbers inside endian-agnostic as well.

One issue preventing further development of this is that I haven't got access to any big-endianness machines. Assuming that is the same for everyone else, the point is of course moot, so we probably won't do anything on this issue for the time being.

Related to endianness is that representation of signed integers might not be portable, so we should perhaps always store unsigned ones behind the scenes (by adding ~2^31 to the number for 32bit ints). Similarly, we might want to use unsigned char* rather than char* to represent byte array data.

[willend]

I worked in brain imaging in the early 00's where machines were a mix of ieee-le and ieee-be. Our adopted solution back then was to always read and write in ieee-be. Could we not adopt a similar solution here?

[tkittel]

Yes, that is what I was trying to say in an admittedly cryptic way by the terms "insentive to endianess" and "endian-agnostic". However, doing it will imply a file format change with all the related hassle, and without an actual big endian machine to test on I don't really feel comfortable carrying out the changes right now. So I hope you agree that we postpone it for now (I just wanted to have an open issue to remind us, since it is in principle the right thing to do).

[willend]

Agreed completely