AES-256 ECB file encryption tool for classic Amiga

[modrobert]

AES is based on the Rijndael cipher developed by two Belgian cryptographers, Joan Daemen and Vincent Rijmen, who submitted a proposal to NIST during the AES selection process.

I wrote most of the stuff in the readme.txt so will just add that here.

modrobert @ 2015-02-21

This is an AES-256 ECB cipher file based encryption tool I developed in
C for classic Amiga (68k), but the code is portable so it should compile on
most sane operating systems. Full source code is included in the file archive.

The benefit with this AES-256 implementation compared to others is that
it doesn't allocate any large memory buffers to encrypt/decrypt, so you can
handle large files with little memory and resources on the target system.

When creating a key file for the cipher I recommend using as much of the
32 bytes (256 bits) you can, at least over 7 chars to make it harder to
bruteforce. You can just type ASCII text in the key file, like a password,
but for stronger cipher it is better with "binary bytes" to utilize all bits.
I recommend the FileX hex editor on aminet if you want to hammer the hex codes
straight into the file, its a great tool BTW. ;)

Function: AES-256 ECB encrypt or decrypt a file.
Feature : Encrypted block counter for increased entropy.
Syntax  : aes [option] <key file> <input file> <output file>
Options : -e = encrypt, -d = decrypt.
Result  : 0 = ok, 1 = read error, 2 = write error, 3 = arg error.
Note    : Only the first 32 bytes of the keyfile will be read.

Alternately, just type 'aes' in an Amiga shell where the file is for info on
the command line usage.


Included files:

aes                - 68k binary compiled with vbcc on real Amiga under WB 3.1.
readme.txt         - You are reading it now. :D
source/aes.c       - My file based command line tool to encrypt/decrypt a file.
source/aes256.c    - AES-256 routines modified to use sbox tables for speed.
source/aes256.h    - Header file structs and prototypes.
source/demo.c      - FIPS 197 compliance test written for the AES-256 routines.
source/compile.txt - Command line to compile the source on Amiga with vbcc.


Changelog:
v1.20 - Added block numbering to encrypted file for increased cipher entropy.
        AES-256 buffer: [ <block number 4 bytes> <plaintext data 12 bytes> ]
v1.01 - Fixed typo, this is ECB, not CBC, thanks Piru.
v1.00 - Initial release.


Background:

The lightweight AES-256 C framework is developed by Ilya Levin and Hal Finney,
and is well suited for the purpose. More information about the authors and
license in the source code. This minimalistic implementation is impressive 
compared to most other stuff out there which usually involves bloated
libraries, strange licenses, or just questionable functionality in general.
There is demo code included you can compile to test that it meets the FIPS-197
requirements according to their advanced cryptographic standard.

Have fun!

Let me know if you find any bugs.

EDIT:

After checking some more; the block counter in plaintext for increased entropy can be regarded as a partially implemented CTR.

Attached file SHA256 hash: 015aff195027efa79360d2cabf71b79b1f58655d41e27c1adadbbe94aa6c7fe3

[alexh]

Cool, I design the AES-256 in SED's (self encrypting drives). We use ECB and XTS we don't do CBC anymore

We also do SHA-256 for salting keys.

Nice to see something new.

[Jope]

Whenever you do modifications to crypto algos, you better know your stuff. I'll wait for some peer review.. ;-)

[modrobert]

I thought the disclaimer took care of that?

[Jope]

I pointed Piru towards this, as he's done cryptanalysis quite often..

< Piru> now lesson to everyone thinking of creating their own cryptographic software
< Piru> it's far too easy to mess up royally. better keep using the industry standard tools and methods
< Piru> case study: http://eab.abime.net/showthread.php?t=77078
< Piru> I have a secret picture called secret.pnm. lets encrypt it
< Piru> lets be generous and use totally random key!
< Piru> openssl rand 32 > key
< Piru> aes -e key secret.pnm encrypted.pnm
< Piru> so we're secure now. right?
< Piru> well except that:
< Piru> echo -e "P6\n1280 687\n255" | dd conv=notrunc of=encrypted.pnm
< Piru> pnmtopng < encrypted.pnm > you-failed-at-crypto.png
< Piru> https://sintonen.fi/pics/you-failed-at-crypto.png
< Piru> Problems: 
< Piru> - It's ECB not CBC.
< Piru> - It's missing HMAC.
< Piru> How to fix it:
< Piru> - Use CBC mode with cryptographically strong random IV
< Piru> - Add HMAC calculated over the IV + cipherdata
< Piru> - At decrypt verify the HMAC before performing decrypt
< Piru> ...or just use "openssl enc" like everyone else.
< Piru> aaaand this concludes the cryptolesson.

https://sintonen.fi/temp/you-failed-at-crypto.txt
https://sintonen.fi/pics/you-failed-at-crypto.png

[modrobert]

I've seen that repeating pattern weakness regarding images before, but not an actual solution to it, assumed it was one of the AES weaknesses.

Ask Piru if he has that image file in its original form, I want to test it with OpenSSL.

EDIT: Never mind, found it here:

https://en.wikipedia.org/wiki/Block_...codebook_(ECB)

EDIT2: Updated the info about this being AES-256 ECB, not CBC. Thanks Piru. I will make some changes and release an update shortly. OK, attachment in first post is now v1.01 where it states ECB instead of CBC.

EDIT3:

I confirmed OpenSSL sucked pretty bad regarding this AES-256 ECB weakness also.

openssl enc -aes-256-ecb -in openssl_test.pnm -out openssl_aes256_ecb_test.pnm
echo -e "P6\n640 400\n255" | dd conv=notrunc of=openssl_aes256_ecb_test.pnm
pnmtopng < openssl_aes256_ecb_test.pnm > openssl_aes256_ecb_test.png

Before AES-256 ECB encryption:
https://modrobert.ddns.net/pics/openssl_test.png

After AES-256 ECB encryption:
http://modrobert.ddns.net/pics/opens...6_ecb_test.png

My mistake was that I misinterpreted the AES routines as CBC, when it actually was ECB.

Will focus on trying to improve my tool now instead, got MD5 running pretty fast on Amiga, wondering if that is good enough for the HMAC. I suspect SHA is way too demanding (as in slow).

[Leffmann]

Pretty interesting to see those images and how they retain enough information to be able to clearly read the text.

Does it work equally well (or bad...) with other types of uncompressed data that can take a bit of wrangling, f.ex. integer WAV data?

Originally Posted by modrobert

Will focus on trying to improve my tool now instead, got MD5 running pretty fast on Amiga, wondering it that is good enough for the HMAC. I suspect SHA is way too demanding (as in slow).

I've written an implementation of SHA-256 in machine language that you could try:
http://aminet.net/package/util/cli/sha256

[modrobert]

Leffmann,

I had a quick look at your SHA-256 routines, was really impressed, and written in assembler which is perfect for this project. Thanks a lot, will try your code soon, and feel free to change mine.

Before reading your post I was thinking about coding the XOR block chain routines for CBC in assembler, which should be easy enough for me, hehe.

Since the IV only needs to be randomized for the first block in CBC it doesn't matter if a bit slow, only matters if incredibly slow. I'm reading up on the "Blum-Blum-Shub" pseudo random generator at the moment.

Regarding the images and AES-256 ECB, I'm a bit sceptic to how useful this weakness really is for an attacker (it looks impressive when trying to discredit methods though, hehe). I would be more worried to see actual fragments of plain text data appearing in the stream than broad formations which clearly is a result of the small 16 byte block size and repetition using the same key.

However, imagine if chaining the data actually introduces other weaknesses. Sure, the image test looks all random static and people are happy, but you might have weakened the whole system by doing so.

[Jope]

Hi,

Originally Posted by modrobert

Will focus on trying to improve my tool now instead, got MD5 running pretty fast on Amiga, wondering if that is good enough for the HMAC. I suspect SHA is way too demanding (as in slow).

Piru asked me to reply:

"Yes. Actually this it the only situation where MD5 can still be considered safe. For any other uses MD5 should be considered wholly insecure."

[modrobert]

Conclusion:

After checking some more, to add cryptographically strong randomization required by the IV for CBC will take this project pretty far from being lightweight, either depending on double precision math libraries, or other hardware/human device to generate the random IV data. The memory requirements would also increase, slower processing in general, not to mention CBC will introduce new weaknesses unless further algorithms are used to pad the cipher data, all leading to a level of bloat which makes it less suitable for classic Amiga hardware.

Also, AES-256 CBC with HMAC seems more appropriate to use for client/server based communication where HMAC protects the integrity of the data, eg. to prevent "man in the middle" attacks (MITM). If you don't care about data integrity, and only want some level of obfuscation on a flat file, then there is no need for HMAC, or am I missing something?


This tool currently provides aes-256-ecb, which seems to work as intended, I will leave it at that. If you want some form of file integrity and improved visual entropy in the encrypted data I recommend storing the unencrypted (plaintext) data in an lha or zip file and then encrypt the archived file.

[Thorham]

Originally Posted by Leffmann

Pretty interesting to see those images and how they retain enough information to be able to clearly read the text.

That's probably because of using the algorithm in the wrong way or a bug.

Originally Posted by modrobert

This tool currently provides aes-256-ecb, which seems to work as intended, I will leave it at that. If you want some form of file integrity and improved visual entropy in the encrypted data I recommend storing the unencrypted (plaintext) data in an lha or zip file and then encrypt the archived file.

You shouldn't have to. When encryption algorithms are used properly and contain no bugs, you can use any file, encrypt it and get output that looks completely random.

[modrobert]

Originally Posted by Thorham

You shouldn't have to. When encryption algorithms are used properly and contain no bugs, you can use any file, encrypt it and get output that looks completely random.

The algorithm is doing what it is supposed to. If you test with 'openssl enc -aes-256-ecb -in plaintext -out ciphertext' you get similar results regarding the "raw image weakness" (as in reduced larger than block size entropy). The differences being how openssl uses the first 16 bytes as a header, which I personally think is less secure since it pretty much acknowledges cipher is being used by just looking at the unecrypted first part in the header of ciphertext, and that openssl salts the password/key.

This is how openssl does it in the header of "encrypted data" (ciphertext).

00000000   53 61 6C 74  65 64 5F 5F  B8 4F 35 94  87 EC 8A B7  Salted__.O5.....

I don't use a header at all, instead the padding is done by encrypting one byte in the "empty" part at the end of file when "filesize % 16 != 0" including plaintext offset, when matching size it just adds one zero byte to the end of file as a marker making file size odd. Simple, yet less revealing. Too bad I mentioned it here, because tried pretty hard to obfuscate what is going on in the source code.

You can question using ECB for the purpose, but I have my doubts about using CBC as an alternative after reading about a weakness regarding XOR chaining all blocks, where sub sequent block plaintext bits can be derived from changing bits in the adjacent one without having the key by bruteforce decrypting the file. Sure, CBC gives you nice random looking entropy when viewing the file, but you have weakened the system by chaining it with XOR, and in effect created an unintended "Turing machine" of sorts to aid clever attackers. You can avoid the weakness if the "server" (program) is running in your control and has additional layers of session security by adding and verifying HMAC, but not when the whole flat file (ciphertext) is available to the attacker using a custom designed program completely in his/her control where the HMAC validation can be bypassed.

[Thorham]

To modrobert:

From what I understand is that a crypto algorithm gets a key as an input, generates a sequence of secure pseudo random numbers equal in length to the data you want to encrypt, and then XORs the sequence with the data. The security should rely entirely on the secureness of the pseudo random numbers generated. The stronger the sequence, the better the encryption.

Perhaps this is the approach you should take.

Side note: I'm not a crypto specialist

Edit:

After looking at the encrypted example image in more detail, I noticed repeating patterns. Look at the s characters in OpenSSL (magnify them). You'll see that they're identical pixel for pixel. This means that this encryption method is broken. Repeating data in a file should not be encrypted in the same way. If you have a text file with ten times 'foobar' in it, then each of the foobars should be different.

Originally Posted by modrobert

Too bad I mentioned it here, because tried pretty hard to obfuscate what is going on in the source code.

Security through obfuscation is not security. Security comes from an algorithm being sound. It shouldn't matter one bit if people understand the code completely.

[modrobert]

Originally Posted by Thorham

To modrobert:

From what I understand is that a crypto algorithm gets a key as an input, generates a sequence of secure pseudo random numbers equal in length to the data you want to encrypt, and then XORs the sequence with the data. The security should rely entirely on the secureness of the pseudo random numbers generated. The stronger the sequence, the better the encryption.

Perhaps this is the approach you should take.

This is how added CBC works, not AES in itself, but as I mentioned previously it introduces a new weakness besides running slow on a bog standard Amiga.

Side note: I'm not a crypto specialist

You could be by simply researching this specific case before replying.

After looking at the encrypted example image in more detail, I noticed repeating patterns. Look at the s characters in OpenSSL (magnify them). You'll see that they're identical pixel for pixel. This means that this encryption method is broken. Repeating data in a file should not be encrypted in the same way. If you have a text file with ten times 'foobar' in it, then each of the foobars should be different.

It is what it is, that is AES-256-ECB, as mentioned previously, openssl will produce the same results except that it mangles the key differently and adds a header. The weakness comes from AES using 16 byte blocks, and when the same key is used on each block in a file it will make "larger than 16 byte formations" possible to view as a raw image in encrypted form, still, the individual 16 byte blocks are encrypted.

Security through obfuscation is not security. Security comes from an algorithm being sound. It shouldn't matter one bit if people understand the code completely.

This is bullshit, out in the wild, in the real world, obfuscation is important. I didn't design this tool using an existing known cipher like AES for academic reasons which only purpose is to be peer reviewed. If you create your own cryptographic method, and want it tested academically, then I agree about your reasoning.

There are so many cipher weaknesses published out there, if you search around and actually read the released papers. The attackers are clearly ahead in the game. You will realize that almost nothing is secure unless you pay a lot of attention exactly how to implement and layer existing methods on case to case basis, and even then it is not completely secure, nothing really is.

[Thorham]

Originally Posted by modrobert

The weakness comes from AES using 16 byte blocks, and when the same key is used on each block in a file it will make "larger than 16 byte formations" possible to view as a raw image, still, the individual 16 byte blocks are encrypted.

Indeed, AES-ECB is insecure, and if you want good encryption then you should use something else.

Originally Posted by modrobert

This is bullshit, out in the wild, in the real world, obfuscation is important.

http://en.wikipedia.org/wiki/Security_through_obscurity

Don't use AES in ECB mode. Pick a mode that's secure even if it's slow on 68k. What's more important, speed or security?

Also, OpenSLL implements several algorithms. Both cyphers and hashes. Some of these are secure and some are not. An example would be SHA2 and MD5 hashes. SHA2 is still secure enough, while MD5 is completely broken. The insecure ones are probably only in OpenSLL for compatibility. Don't use them, use the secure ones.

[modrobert]

Originally Posted by Thorham

Indeed, AES-ECB is insecure, and if you want good encryption then you should use something else.

Like what? That will actually run on a classic Amiga.

As I mentioned several times previously, the feasible CBC method relying on XOR has a serious weakness. To quote myself from previous post: "You can question using ECB for the purpose, but I have my doubts about using CBC as an alternative after reading about a weakness regarding XOR chaining all blocks, where sub sequent block plaintext bits can be derived from changing bits in the adjacent one without having the key by bruteforce decrypting the file. Sure, CBC gives you nice random looking entropy when viewing the file, but you have weakened the system by chaining it with XOR, and in effect created an unintended 'Turing machine' of sorts to aid clever attackers. You can avoid the weakness if the "server" (program) is running in your control and has additional layers of session security by adding and verifying HMAC, but not when the whole flat file (ciphertext) is available to the attacker using a custom designed program completely in his/her control where the HMAC validation can be bypassed."

Don't use AES in ECB mode. Pick a mode that's secure even if it's slow on 68k. What's more important, speed or security?

It's only secure in the sense that you confuse visual randomness with security.

Also, OpenSLL implements several algorithms. Both cyphers and hashes. Some of these are secure and some are not. An example would be SHA2 and MD5 hashes. SHA2 is still secure enough, while MD5 is completely broken. The insecure ones are probably only in OpenSLL for compatibility. Don't use them, use the secure ones.

Yes, and OpenSSL for Amiga 68k requires 8mb RAM, WB 3.x, and relies on an additional random device handler. I created this tool as a lightweight alternative, as long as the user is aware that it is AES-256-ECB and take precautions it can be used.

My previous advice to encrypt an already compressed file archive like lha or zip instead of raw image data pretty much gives you the entropy and integrity provided by the flawed AES-256-CBC with "encrypt-then-HMAC", and the added bonus of making the encrypted file smaller.

[Thorham]

Originally Posted by modrobert

Like what? That will actually run on a classic Amiga.

If you want to use AES, then you have to pick the right mode for the job. Here's an interesting post on StackOverflow:

http://stackoverflow.com/questions/1...cb-ctr-ocb-cfb

You should also search for 'aes mode' in the cryptography section of StackExchange:

http://crypto.stackexchange.com/

Originally Posted by modrobert

It's only secure in the sense that you confuse visual randomness with security.

I don't confuse visual randomness with security, nor do I confuse it with good quality non-crypto pseudo-randomness. I simply spotted the problem with ECB mode in the example image you provided.

Originally Posted by modrobert

My previous advice to encrypt an already compressed archive like lha or zip file instead of raw image data pretty much gives you the entropy and integrity provided by the flawed AES-256-CBC using HMAC.

Don't use ECB. ECB is not for encrypting multiple blocks.

[modrobert]

I don't "have to" do anything, and no one "have to" use this against their own will, I already linked OpenSSL in my previous post, the world is full of choices. I have explained my motivations thoroughly. So, what's up with the broken record routine?

Play nice or I will design my own cryptographic method and post here, imagine the shit storm of posts that would produce with people who advice against it.

Any random advice posted online should be taken as law, especially wikipedia, right? Without empirical tests, just tell everyone else the same thing like a parrot. Did you know I edit some of the wikipedia pages? Anyone can edit, and if a person appears to be correct the info is rarely removed or edited. Scary thought, you might actually be linking me my own advice.

I was really eager to implement CBC on my own because it would mean writing the XOR routines in assembler and linking it with the rest, and was happy about leffmann's SHA256 for potential use to generate HMAC, but then I hit the first hurdle, to add a cryptographically strong random routine was harder than expected, and cost a lot of system resources to do right when thinking Amiga classic terms. Sure, you can just wing it and grab some custom chipset register bytes and hope for the best, but then it wouldn't really be CBC, would it?

Then came the worst part, the weakness I mentioned before which at first seemed hard to accomplish, but later when going deeper into research material turned out to be pretty awful and requiring relatively few loops to gain a whole block of plaintext data so you can't really call it "brute force", more like "served on a platter", thank you CBC.

If you search around almost every page hit will tell you to use CBC instead of ECB, and they will either show the Tux image appearing on ciphertext to quote wikipedia or some other picture to prove their point. No one seems to even bother mentioning that each block is still secure with ECB, as in not breached yet, even if the pattern appears. I'm aware there are attack possibilities with similarities on the blocks, but being aware of the weakness makes it less of a problem, and it is still a hell of a lot better than plaintext.

I rather have block similarities with ECB, which is still functioning at the core, than tricking people into feeling safe with CBC just because half of internet thinks it is best.

[Thorham]

To modrobert:

There are other AES modes than just ECB and CBC. This is why I said to use the right mode for the job. Both ECB and CBC have problems, but they're not your only choice.

Here's a nice quote from http://tonyarcieri.com/all-the-crypt...robably-broken

If you find yourself reaching for any form of encryption that isn’t an authenticated encryption mode, you’re probably doing it wrong. You shouldn’t ever be choosing between CBC or CFB or CTR (or god forbid ECB). Unless you’re a cryptographer, these should be considered dangerous low-level primitives not for the consumption of mere mortals.

Encryption seems to be tricky business, and I think you may be underestimating the difficulty of getting it right. For hashes it's easy because there are no modes to pick and there's no need for authentication. Simply implement them according to spec and you're done. Did this myself for SHA1 and SHA2 (256). After reading about encryption however, I don't think it's that clear cut and dry.

As for generating randomness on Amiga hardware, probably not possible to be truly secure. For non-secure applications that simply need random seeds, I use a combination of hardware registers and SHA1 or SHA2 (to remove bias). For SHA2 I simply add the contents of eight hardware register bytes to each other, write the resulting byte to the SHA2 input buffer, and repeat this for all 64 bytes in the SHA2 input buffer. Then I hash the buffer. Great for getting entropy that's ten million times better than a time stamp. Whether or not it's secure is anyone's guess (but security wasn't the goal for this).

[modrobert]

Thorham,

Thanks for the tips about random seed.

In my opinion the attitude about cryptography should be; anything is better than plaintext.

You could always experiment by adding daredevil command line option, eg. --this-shit-might-fail-badly, to make the user aware they are bleeding edge alpha testing.

So, thinking along these lines, in order to stop the state-machine XOR back and forth attack on CBC, what do you think would be a way forward?

I have an idea about keeping XOR in CBC, but instead of doing XOR on the plaintext data before encrypting, do it on a checksum/hash of the plaintext data block instead. Perhaps that could prevent the unintended state-machine revealing block data, all the attacker would get is hash bits then.