How can I encrypt data in chunks with AES-128-CBC?

[PabloACZ]

I'm currently doing a modification of BlueDump that fixes most problems present in the original application, and I want to add support for contents bigger than 45MB (like those present in the "Wii & The Internet" Channel that comes pre-installed in every new Wii).

The problem is that, in order to add support for those contents, I need to change the whole dumping process to make it write data in chunks instead of allocating memory to copy the full content data to MEM2 (yep, that's what the application originally did).

Currently, this is the code I'm using:

Code: Select all

u32 GetBIGContent(FILE *f, u64 id, u16 content, u16 index, u32 size)
{
	char path[ISFS_MAXPATH];
	
	sprintf(path, "/title/%08x/%08x/content/%08x.app", TITLE_UPPER(id), TITLE_LOWER(id), content);
	logfile("Regular content path is '%s'.\n", path);
	printf("Adding regular content %08x.app... ", content);
	
	s32 fd = ISFS_Open(path, ISFS_OPEN_READ);
	if (fd < 0)
	{
		//printf("ISFS_Open for '%s' returned %d.\n", path, fd);
		logfile("ISFS_Open for '%s' returned %d.\n", path, fd);
		return 0;
	}
	
	u32 blksize = BLOCKSIZE; // 16KB
	
	u8 *buffer = (u8*)memalign(32, blksize);
	if (buffer == NULL)
	{
		//printf("Allocating memory for buffer failed.\n");
		logfile("Allocating memory for buffer failed.\n");
		ISFS_Close(fd);
		Unmount_Devices();
		Reboot();
	}
	
	u8 *encryptedcontentbuf = (u8*)memalign(32, blksize);
	if (encryptedcontentbuf == NULL)
	{
		//printf("Allocating memory for buffer failed.\n");
		logfile("Allocating memory for encryptedcontentbuf failed.\n");
		ISFS_Close(fd);
		Unmount_Devices();
		Reboot();
	}
	
	u32 i, pad, size2 = 0;
	s32 ret = 0;
	
	logfile("Writing...\n");
	for (i = 0; i < size; i += blksize)
	{
		if (blksize > size - i)
		{
			blksize = size - i;
		}
		
		ret = ISFS_Read(fd, buffer, blksize);
		if (ret < 0) break;
		
		/* Pad data to a 16-byte boundary (required for the encryption process). Probably only needed for the last chunk */
		if ((blksize % 16) != 0)
		{
			pad = 16 - blksize % 16;
			memset(&(buffer[i+blksize]), 0x00, pad);
			logfile("Content chunk #%u padded to 16-byte boundary. Current blksize: %u.\n", (i / 0x4000), blksize);
			blksize += pad;
		}
		
		encrypt_buffer(index, buffer, encryptedcontentbuf, blksize);
		
		/* Pad data to a 64-byte boundary (required for the WAD alignment). Again, probably only needed for the last chunk */
		if ((blksize % 64) != 0)
		{
			pad = 64 - blksize % 64;
			memset(&(encryptedcontentbuf[i+blksize]), 0x00, pad);
			logfile("Encrypted content chunk #%u padded to 64-byte boundary. Current blksize: %u.\n", (i / 0x4000), blksize);
			blksize += pad;
		}
		
		fwrite(encryptedcontentbuf, 1, blksize, f);
		
		size2 += blksize;
	}
	
	logfile("Content added successfully. Original content size: %u bytes. size2: %u bytes.\n", size, size2);
	
	free(buffer);
	free(encryptedcontentbuf);
	ISFS_Close(fd);
	
	if (ret < 0)
	{
		//printf("Failed to read data into content buffer.");
		logfile("Failed to read data into content buffer.");
		fclose(f);
		Unmount_Devices();
		Reboot();
	}
	
	printf("done.\n");
	
	header->data_len += size2;
	return size2;
}

When I unpack the dumped title, the SHA-1 hash of the decrypted contents do not match those present on the TMD file. Please, if you have any idea, it would be greatly appreciated if you say it! Thanks in advance.

[tueidj]

Code: Select all

memset(&(buffer[i+blksize]), 0x00, pad);

Code: Select all

memset(&(encryptedcontentbuf[i+blksize]), 0x00, pad);

Both of those lines are wrong, you are memsetting beyond the end of the buffers (remove "i+").

[PabloACZ]

Fixed, thanks for the correction. But still no luck about the content verification, the hashes are different. :/

EDIT: Well, it seems like I have to save the last 16 bytes of every new chunk and use that as the IV for the next one, according to some info about Cipher-block chaining I just read. I'll give it a try.

[tueidj]

I wondered about that but there was no IV parameter for encrypt_buffer(), so I figured it was managed internally. Guess not.

[PabloACZ]

I wondered about that but there was no IV parameter for encrypt_buffer(), so I figured it was managed internally. Guess not.

There actually is, the index value represents the Content Index stored on the TMD for the current content. The encrypt_buffer() function pads it to 16 bytes with zeros.

btw, still no luck... I'm no longer using the encrypt_buffer() function. The current IV is managed directly with each loop.

[PabloACZ]

Sorry for the double post. I did it!

The hashes from the decrypted contents now match those encountered on the TMD. Thanks for all your help, tueidj.

To anyone else: the SVN repository is located at http://code.google.com/p/bluedump-mod. Check it out.

[BenoitRen]

Congratulations! It's always awesome when something you're programming finally works as intended!