English Amiga Board


Go Back   English Amiga Board > Coders > Coders. General > Coders. Releases

 
 
Thread Tools
Old 19 February 2015, 18:57   #21
Thorham
Computer Nerd

Thorham's Avatar
 
Join Date: Sep 2007
Location: Rotterdam/Netherlands
Age: 41
Posts: 2,964
Quote:
Originally Posted by modrobert View Post
what do you think would be a way forward?
To pick the right AES mode and use it properly. You seem completely fixated on ECB and CBC. There are more modes of operation than just these two. If I were you, I'd do some more research. You could start with this university course about cryptography (registration is free and simple) https://www.coursera.org/course/crypto

Also, did you read through the article I linked to? Here it is again http://tonyarcieri.com/all-the-crypt...robably-broken

READ IT

Like I said before, cryptography is tricky business and all too easy to get wrong. This says nothing about ones abilities, but rather something about the subject matter. Don't underestimate it.
Thorham is offline  
AdSense AdSense  
Old 19 February 2015, 20:57   #22
modrobert
old bearded fool

modrobert's Avatar
 
Join Date: Jan 2010
Location: Bangkok
Age: 50
Posts: 414
I have checked, and CBC is by far the least resource demanding (besides ECB). As mentioned before, see my posts before you joined the discussion in the thread.

Don't be shy, give me some suggestions to secure CBC. What do you think of my idea? I want your opinion, the establishment does not interest me, only results.
modrobert is offline  
Old 19 February 2015, 21:57   #23
Thorham
Computer Nerd

Thorham's Avatar
 
Join Date: Sep 2007
Location: Rotterdam/Netherlands
Age: 41
Posts: 2,964
Quote:
Originally Posted by modrobert View Post
I want your opinion, the establishment does not interest me, only results.
That's quite odd, because the establishment is undoubtedly quite knowledgeable about this topic. If you want quality results, then using the available knowledge is the best way to go about this. Asking for the opinion of a crypto n00b (me) is most certainly not the right way.

If you're serious about this, then the only practical way is to take your time and research this topic extensively, and use what are considered proper methods.
Thorham is offline  
Old 20 February 2015, 04:11   #24
modrobert
old bearded fool

modrobert's Avatar
 
Join Date: Jan 2010
Location: Bangkok
Age: 50
Posts: 414
How would you ever invent something new if you always used what was considered proper established methods?
modrobert is offline  
Old 20 February 2015, 05:36   #25
Thorham
Computer Nerd

Thorham's Avatar
 
Join Date: Sep 2007
Location: Rotterdam/Netherlands
Age: 41
Posts: 2,964
Quote:
Originally Posted by modrobert View Post
How would you ever invent something new if you always used what was considered proper established methods?
How can you invent anything new that's any good if you're new to something, and not willing to look at what has been done already?

You're discarding decades of knowledge about this topic and I want to know why.
Thorham is offline  
Old 20 February 2015, 07:09   #26
modrobert
old bearded fool

modrobert's Avatar
 
Join Date: Jan 2010
Location: Bangkok
Age: 50
Posts: 414
I'm not discarding anything, on the contrary.

Lets recap:

You are replying to a thread I started by sharing an open source AES program for classic Amiga.

You claim to be, using your own words, a crypto n00b, and yet you are treating me as if I would need lessons on the subject while constantly arguing my decisions are wrong. Which does feel a bit strange, not only due to the fact I'm older than you, but also the irony of it.

When I try to turn the discussion into something constructive, you back off, and keep quoting other people and their suggestions.

What is your purpose in this forum thread?

Last edited by modrobert; 20 February 2015 at 07:16.
modrobert is offline  
Old 20 February 2015, 08:58   #27
Thorham
Computer Nerd

Thorham's Avatar
 
Join Date: Sep 2007
Location: Rotterdam/Netherlands
Age: 41
Posts: 2,964
To modrobert:

If you're unwilling to do the research that's required to make proper encryption software, then that's fine by me.

One more thing: Don't pull that age crap. It's ridiculous.
Thorham is offline  
Old 20 February 2015, 12:29   #28
modrobert
old bearded fool

modrobert's Avatar
 
Join Date: Jan 2010
Location: Bangkok
Age: 50
Posts: 414
Quote:
Originally Posted by Thorham View Post
One more thing: Don't pull that age crap. It's ridiculous.
Yes, that was a bit low, have to admit, older doesn't necessarily mean wiser.


I have a new idea to deal with the lack of entropy in ECB, just serialize the plaintext data for each block before encryption, like this:

[ <block number 4 bytes> <plaintext data 12 bytes> ]

The benefit is that's I'm not actually modifying the cryptographic method in itself, and the nature of AES should take care of the entropy by adding a few bytes which are guaranteed to be unique for each block, can't wait to test it.

EDIT:

Hmm, second thought, this would mean the first 4 bytes will be known in each block, at least if the attacker is aware of the method used for the input data. On the other hand, the attackers know how XOR is implemented by chaining blocks in CBC, which was even worse as it turned out. The advantage of serializing plaintext blocks as opposed to conventional hashing is that no additional data has to be stored outside the encrypted data, eg. by padding or using header.

Many times the attacker has information about the plaintext data, especially when related to open source software, either by protocol encapsulated within the encryption or by other means, so not sure if I should care about that or not.

EDIT2:

Behold, I'm actually resorting to quoting someone else.

Quote:
AES-256 has sustained 15 years of cryptanalysis, and it can be stated that no knowledge of some plaintext bytes would help to reveal the other bytes no matter what mode of operation (CBC, CTR, etc.) is used.
So, will go ahead and implement it.

Last edited by modrobert; 20 February 2015 at 14:05.
modrobert is offline  
Old 21 February 2015, 00:26   #29
modrobert
old bearded fool

modrobert's Avatar
 
Join Date: Jan 2010
Location: Bangkok
Age: 50
Posts: 414
I have updated the first post with aes_v1_20.lha.

Look, we have entropy!



Let me know if you find any bugs.
modrobert is offline  
Old 23 February 2015, 05:45   #30
Thorham
Computer Nerd

Thorham's Avatar
 
Join Date: Sep 2007
Location: Rotterdam/Netherlands
Age: 41
Posts: 2,964
Quote:
Originally Posted by modrobert View Post
Yes, that was a bit low, have to admit, older doesn't necessarily mean wiser.


After thinking about this for a bit, I came up with some ideas for improving ECB and CBC. Here's how I would use them:

ECB:

1. Generate initialization vector -> IV.
2. SHA256 password -> key.
3. Join IV and key and run through SHA256 -> newKey.
4. Encrypt block of plain text with newKey.
5. Run newKey through SHA256 -> newKey.
6. Goto 4 until done.

CBC:

1. Generate initialization vector -> IV.
2. SHA256 password -> key.
3. Encrypt block of plain text with key.
4. Run key through SHA256 -> key.
5. Goto 3 until done.

In both cases SHA is used without it's normal padding, because the keys and IVs are all the same size as the SHA256 input buffer (for AES256).

Generation of IV:

1. Add together eight bytes from hardware registers.
2. Write resulting byte to SHA256 buffer.
3. Goto 1 until 64 bytes have been written to SHA buffer.
4. Hash buffer with SHA256.

Generating the IV is very slow because reading the involved registers is slow ($dff000 range registers are slow, CIA registers are VERY slow), but it only has to be done once for each file that you want to encrypt, so it doesn't matter.

On top of that, authentication is needed to prevent modifying the encrypted data in an undetectable way.

In both cases the improved security should come from the fact that the key changes for each encrypted block. For ECB, the added IV should also help.

Because I'm not a cryptography expert in any way, shape or form, I don't know how good these two schemes truly are, but at the surface they seem fine. However, I wouldn't bet national security on them
Thorham is offline  
Old 23 February 2015, 10:25   #31
modrobert
old bearded fool

modrobert's Avatar
 
Join Date: Jan 2010
Location: Bangkok
Age: 50
Posts: 414
Quote:
Originally Posted by Thorham View Post
ECB:

1. Generate initialization vector -> IV.
2. SHA256 password -> key.
3. Join IV and key and run through SHA256 -> newKey.
4. Encrypt block of plain text with newKey.
5. Run newKey through SHA256 -> newKey.
6. Goto 4 until done.
1) What makes the "newKey" change between each block/run in step 5?

2) How will the IV be stored for decryption?


Quote:
CBC:

1. Generate initialization vector -> IV.
2. SHA256 password -> key.
3. Encrypt block of plain text with key.
4. Run key through SHA256 -> key.
5. Goto 3 until done.
1) What makes the "key" change between each run/block in step 4 (assuming you actually mean newKey as in the ECB case)?

2) How is the IV stored for decryption?

3) Where in the sequence is XOR performed?


Quote:
Generation of IV:

1. Add together eight bytes from hardware registers.
2. Write resulting byte to SHA256 buffer.
3. Goto 1 until 64 bytes have been written to SHA buffer.
4. Hash buffer with SHA256.

Generating the IV is very slow because reading the involved registers is slow ($dff000 range registers are slow, CIA registers are VERY slow), but it only has to be done once for each file that you want to encrypt, so it doesn't matter.
I think it is acceptable to use CIA registers in terms of random seed. However, this is where the source code goes completely Amiga specific and in effect loses its portability towards other resource constrained platforms, which is OK, just noting the fact.


Quote:
On top of that, authentication is needed to prevent modifying the encrypted data in an undetectable way.
How can authentication be made unavoidable? In this case, with a flat file, the attacker can just use (modify) decryption code which ignores the authentication. This is not a client-server based implementation where we still have control over decryption execution.


Quote:
In both cases the improved security should come from the fact that the key changes for each encrypted block. For ECB, the added IV should also help.
Regardless of method used to change the AES key between each block (loop run), from that point on AES is no longer relying on a secret key, rather the integrity of the method used to hash it. I assume that is the reason most block based methods never change the AES key itself, but rather add IV based hashing to the mix or chain data between blocks.

Last edited by modrobert; 23 February 2015 at 11:00.
modrobert is offline  
Old 23 February 2015, 11:09   #32
Thorham
Computer Nerd

Thorham's Avatar
 
Join Date: Sep 2007
Location: Rotterdam/Netherlands
Age: 41
Posts: 2,964
Quote:
Originally Posted by modrobert View Post
1) What makes the "newKey" change between each block/run in step 5?
The key is generated by rehashing the hash. Basically, the key is the hash output from SHA. This is hash is simply fed back through SHA. The big issue with this is that SHA might repeat, simply because you get a collision. To make this better, some kind of counter should be hashed with the key, instead of just the key.

Quote:
Originally Posted by modrobert View Post
2) How will the IV be stored for decryption?
From what I understand from the course material I've been watching, the IV can just be stored as is, and shouldn't help the attacker at all. It's only purpose is to make sure that encrypting the same plain text with the same key yields a different cypher text each time. I'll watch the relevant videos again to see if I got that right.

Quote:
Originally Posted by modrobert View Post
3) Where in the sequence is XOR performed?
The XOR step is the same as in normal CBC. I've only described a method for changing the key for each block.

Quote:
Originally Posted by modrobert View Post
I think it is acceptable to use CIA registers in terms of random seed.
It seems to be the only way.

Quote:
Originally Posted by modrobert View Post
However, this is where the source code goes completely Amiga specific and in effect loses its portability towards other resource constrained platforms, which is OK, just noting the fact.
The IV part should actually be platform specific, because generating random data will be different on different systems. Not a problem.

Quote:
Originally Posted by modrobert View Post
How can authentication be made unavoidable? In this case, with a flat file, the attacker can just use (modify) decryption code which ignores the authentication. This is not a client-server based implementation where we still have control over decryption execution.
Yeah, that's a problem. There are some chapters in the course I've been watching about this. The answer is probably in there.

Quote:
Originally Posted by modrobert View Post
Regardless of method used to change the AES key between each block (loop run), from that point on AES is no longer relying on a secret key, rather the integrity of the method used to hash it.
You still need some kind of key derivation algorithm to generate a strong key from the password.

Quote:
Originally Posted by modrobert View Post
I assume that is the reason most stream based methods never change the AES key itself between blocks, but rather add IV based hashing to the mix or chain data between blocks.
Perhaps, but using a cryptographic hash algorithm does make it extremely hard to make proper guesses about the generated sequence of keys. Heck, I wouldn't be surprised if you could get away with simply incrementing the first long word of the hash by one for each block.

The big question to me seems to be how to go about generating different keys for each AES block. The SHA method I described should be nice (apart from the collision issue) but is obviously slow.

So, yes, this could use some work, but it's only my first idea
Thorham is offline  
Old 23 February 2015, 15:59   #33
modrobert
old bearded fool

modrobert's Avatar
 
Join Date: Jan 2010
Location: Bangkok
Age: 50
Posts: 414
Quote:
Originally Posted by Thorham View Post
The key is generated by rehashing the hash. Basically, the key is the hash output from SHA. This is hash is simply fed back through SHA. The big issue with this is that SHA might repeat, simply because you get a collision. To make this better, some kind of counter should be hashed with the key, instead of just the key.
Another more obvious drawback of changing the key each AES block is that you are adding more keys which can partially decrypt the ciphertext file. In this case, for each MB of plaintext data you introduce 62500 new keys. Granted, each block individually in ciphertext has the same probability of being brute forced, but the integrity of the file as a whole is weakened for each new key.


About SHA256 hashing the whole file, even if the attacker can bypass and decrypt anyway, it still serves a purpose for the user to confirm the file is intact after decryption, and in this case the SHA256 hash can be stored in two additional encrypted blocks (as opposed to plaintext file padding). Eg. file decryption ends with some message like "Decrypted file matches checksum" or "... failed ...".

Last edited by modrobert; 23 February 2015 at 17:46.
modrobert is offline  
Old 26 February 2015, 12:10   #34
Thorham
Computer Nerd

Thorham's Avatar
 
Join Date: Sep 2007
Location: Rotterdam/Netherlands
Age: 41
Posts: 2,964
Something else for a bit. I've translated aes256.c in your archive to assembly language, and made some optimizations (not done with that, yet). Works nicely so far. Should go well with my SHA1 and SHA2 implementations in asm.

Seems that this version is a naive byte based version, though, as the original author explains, which means it won't be the fastest. Perhaps I can get rid of that in my asm version.
Thorham is offline  
Old 26 February 2015, 15:41   #35
modrobert
old bearded fool

modrobert's Avatar
 
Join Date: Jan 2010
Location: Bangkok
Age: 50
Posts: 414
Quote:
Originally Posted by Thorham View Post
Something else for a bit. I've translated aes256.c in your archive to assembly language, and made some optimizations (not done with that, yet). Works nicely so far. Should go well with my SHA1 and SHA2 implementations in asm.
Yes, I'm sure there is a lot of room for optimization using assembler, it's how AES was implemented back in the day. If you look at existing assembler source code, beware that most implementations are little-endian, and not big-endian like Amiga.

Quote:
Seems that this version is a naive byte based version, though, as the original author explains, which means it won't be the fastest. Perhaps I can get rid of that in my asm version.
Actually, it was originally designed to do on-the-fly calculations instead of using lookup tables which is faster, but then one of the authors added the S-Box tables as an option which is enabled with '#define BACK_TO_TABLES' in aes256.c.

I'm guessing the original author did on-the-fly calculations initially rather than lookup tables because the S-Box table can easily be found when doing byte pattern search for reverse engineering purposes, and generally embedded solutions rely on obfuscation, especially since chips can be decapped for keys. Hiding cryptographic method used can delay reverse engineering, anything to make it harder for the competition.
modrobert is offline  
Old 26 February 2015, 16:00   #36
Thorham
Computer Nerd

Thorham's Avatar
 
Join Date: Sep 2007
Location: Rotterdam/Netherlands
Age: 41
Posts: 2,964
Quote:
Originally Posted by modrobert View Post
Yes, I'm sure there is a lot of room for optimization using assembler, it's how AES was implemented back in the day.
Yeah, quite a bit. There's a lot of crufty byte crud that can be replaced. Been doing that just now.

Quote:
Originally Posted by modrobert View Post
Actually, it was originally designed to do on-the-fly calculations instead of using lookup tables which is faster
The lookup tables are indeed a lot faster. They can be made even faster if they're extended to 16 bit, but that increases the table space to 256 kilobytes. Can be done at run time quickly, though.

Quote:
Originally Posted by modrobert View Post
I'm guessing the original author did on-the-fly calculations initially rather than lookup tables because the S-Box table can easily be found when doing byte pattern search for reverse engineering purposes
Actually, it seems that this implementation is a simple, basic reference implementation for people who wanted or needed one. See the original web page.

Quote:
Originally Posted by modrobert View Post
Hiding cryptographic method used can delay reverse engineering, anything to make it harder for the competition.
Good cryptographic methods don't rely on obfuscation. Secrecy through obscurity isn't good. AES doesn't need obscurity. The whole world knows about it, and it hasn't been broken yet.

Secret algorithms are only tested by a handful of people, and probably only for a short period. This means that there are all kinds of possible weaknesses that won't be found. Don't ever rely on this. Really, don't. The best crypto algorithms are the ones that are widely known, and still haven't been broken after years.

For a crypto algorithm to succeed, it's absolutely vital that it's completely open.
Thorham is offline  
Old 27 February 2015, 04:59   #37
modrobert
old bearded fool

modrobert's Avatar
 
Join Date: Jan 2010
Location: Bangkok
Age: 50
Posts: 414
Quote:
Originally Posted by Thorham View Post
Good cryptographic methods don't rely on obfuscation. Secrecy through obscurity isn't good. AES doesn't need obscurity. The whole world knows about it, and it hasn't been broken yet.

Secret algorithms are only tested by a handful of people, and probably only for a short period. This means that there are all kinds of possible weaknesses that won't be found. Don't ever rely on this. Really, don't. The best crypto algorithms are the ones that are widely known, and still haven't been broken after years.

For a crypto algorithm to succeed, it's absolutely vital that it's completely open.
As I mentioned previously (you left that out from my quote for some reason); when an attacker extract keys by decapping chips, which is common and relatively cheap for any hardware these days. Then it doesn't matter if AES, all you have is the overall level of security where obfuscation plays an important part. Obviously this means making it as difficult as possible, designing purposely to mislead, whatever it takes to discourage or delay the attacker.

Hacking from a consumer perspective is not the only reason to secure the devices, most of the big corporations (Sony, Samsung, Apple, etc.) systematically reverse engineer each others hardware designs to use in their own products, and obfuscation is a way of hiding that, to avoid being sued for patent infringement.

Any hardware can be reverse engineered given enough effort, that's pretty much the reality out there.
modrobert is offline  
Old 27 February 2015, 11:49   #38
alexh
Thalion Webshrine
alexh's Avatar
 
Join Date: Jan 2004
Location: Oxford
Posts: 11,900
Quote:
Originally Posted by modrobert View Post
when an attacker extract keys by decapping chips
No-one stores keys uncompressed in plain text in silicon. Even if you de-capped any of the chips I've made with AES in them the Keys are programmed in during authentication.

That is where hackers target these days. Authentication. They don't decap chips.
alexh is offline  
Old 27 February 2015, 14:53   #39
modrobert
old bearded fool

modrobert's Avatar
 
Join Date: Jan 2010
Location: Bangkok
Age: 50
Posts: 414
Quote:
Originally Posted by alexh View Post
No-one stores keys uncompressed in plain text in silicon. Even if you de-capped any of the chips I've made with AES in them the Keys are programmed in during authentication.

That is where hackers target these days. Authentication. They don't decap chips.
Yes, plaintext would be nice. I know for a fact decapping is still a lucrative business for custom ASICs and microcontrollers, even if the keys are mangled by opcodes and never crosses a bus, you can still map the logic producing the key using a microscope. If this is a device which needs auth via device driver on a computer, then dumping data on a bus could be another option, perhaps in conjunction with decap (to get the other key).

This is a bit off topic though, my point was about obfuscation, there are indeed cases where it can be beneficial.

Last edited by modrobert; 27 February 2015 at 15:49.
modrobert is offline  
Old 02 March 2015, 15:14   #40
Thorham
Computer Nerd

Thorham's Avatar
 
Join Date: Sep 2007
Location: Rotterdam/Netherlands
Age: 41
Posts: 2,964
To modrobert:

If you want a version that's probably faster when compiled, then you'll want to use the OpenSSL version. Everything you need is in crypto\aes\aes_core.c.

I've actually switched to that version, because the optimized version I was working on can't be optimized as well as the OpenSLL version. It's also longer, and requires 256kb of table space, vs 16kb in my optimized OpenSSL version (normally has 8*1024kb tables).

At first I thought it was going to be crappy, but it turns out to be quite nice.
Thorham is offline  
AdSense AdSense  
 


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Similar Threads
Thread Thread Starter Forum Replies Last Post
windows tool to convert ipf to wwp / normal adf file jotd Coders. General 12 08 May 2014 10:02
universe amiga 500 256 colors ! ? turrican3 Retrogaming General Discussion 14 09 April 2014 22:35
How to change default tool for file types AndyFC project.ClassicWB 2 13 February 2013 11:18
Classic OS System.zip file problems FOL project.ClassicWB 2 21 July 2007 18:48
Pal Amiga is 256 redblade Retrogaming General Discussion 9 05 April 2006 17:30

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +2. The time now is 01:09.


Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2017, vBulletin Solutions, Inc.
Page generated in 0.33316 seconds with 13 queries