Serializing any-size objects to a random access file
This article will delve into the details involved in serializing and deserializing arbitrarily sized objects to- and from a byte array. The contents of the byte array are stored in a random access file. An index file is kept to allow for objects of varying size.
Serializable objects
Any object that is an instance of a class that implements the Serializable interface, can be serialized and deserialized. Such a class is e.g. coded as follows:
1import java.io.Serializable;
2public class Entry implements Serializable {
3 ...
4}
Object serialization to- and deserialization from a byte array
To achieve object serialization the ObjectOutputStream class is used, and for object deserialization the ObjectInputStream class. To write a serialized object to an array of bytes the ByteArrayOutputStream class is used, and to read a serialized object from an array of bytes the ByteArrayInputStream class is used.
An example of such serialization and deserialization code is listed below:
1import java.io.ByteArrayInputStream;
2import java.io.ByteArrayOutputStream;
3import java.io.IOException;
4import java.io.ObjectInputStream;
5import java.io.ObjectOutputStream;
6import java.io.Serializable;
7import java.util.Date;
8
9public class Entry implements Serializable {
10
11 private Date timestamp = new Date();
12 private String username;
13 private String message;
14
15 public Entry(String username, String message) {
16 this.username = username;
17 this.message = message;
18 }
19
20 public String getMessage() {
21 return message;
22 }
23
24 public Date getTimestamp() {
25 return timestamp;
26 }
27
28 public String getUsername() {
29 return username;
30 }
31
32 public static byte[] serialize(Entry entry)
33 throws IOException {
34
35 ByteArrayOutputStream baos = new ByteArrayOutputStream();
36 ObjectOutputStream oos = new ObjectOutputStream(baos);
37 oos.writeObject(entry);
38 oos.flush();
39
40 return baos.toByteArray();
41 }
42
43 public static Entry deserialize(byte[] byteArray)
44 throws IOException, ClassNotFoundException {
45
46 ObjectInputStream ois = new ObjectInputStream(new ByteArrayInputStream(byteArray));
47 Entry entry = (Entry) ois.readObject();
48
49 return entry;
50 }
51}
Append to- and read from random access file
Now we have the serialized object in a byte array we can store it in a random access file and read it back later. Java's highly speed optimized new I/O library (nio) is used for this. Because the length of the serialized object can vary, an index file is kept, which allows for arbitrarily sized serialized objects to be stored.
The append to- and read from file code is listed below:
1import java.io.FileNotFoundException;
2import java.io.IOException;
3import java.io.RandomAccessFile;
4import java.nio.ByteBuffer;
5import java.nio.channels.FileChannel;
6
7public class EntryFile {
8
9 public static final String DATA_EXT = ".dat";
10 public static final String INDEX_EXT = ".idx";
11 public static final int INDEX_SIZE = Long.SIZE / 8;
12 private FileChannel fci;
13 private FileChannel fcd;
14
15 public EntryFile(String fileName)
16 throws FileNotFoundException, IOException {
17
18 fci = new RandomAccessFile(fileName + EntryFile.INDEX_EXT, "rw").getChannel();
19 fci.force(true);
20 fcd = new RandomAccessFile(fileName + EntryFile.DATA_EXT, "rw").getChannel();
21 fcd.force(true);
22 }
23
24 public void close()
25 throws IOException {
26
27 fcd.close();
28 fci.close();
29 }
30
31 public long appendEntry(Entry entry)
32 throws IOException {
33
34 // Calculate the data index for append to data
35 // file and append its value to the index file.
36 long byteOffset = fci.size();
37 long index = byteOffset / (long) EntryFile.INDEX_SIZE;
38 long dataOffset = (int) fcd.size();
39 ByteBuffer bb = ByteBuffer.allocate(EntryFile.INDEX_SIZE);
40 bb.putLong(dataOffset);
41 bb.flip();
42 fci.position(byteOffset);
43 fci.write(bb);
44
45 // Append serialized object data to the data file.
46 byte[] se = Entry.serialize(entry);
47 fcd.position(dataOffset);
48 fcd.write(ByteBuffer.wrap(se));
49
50 return index;
51 }
52
53 public Entry readEntry(long index)
54 throws IOException, ClassNotFoundException {
55
56 // Get the data index and -length from the index file.
57 long byteOffset = index * (long) EntryFile.INDEX_SIZE;
58 ByteBuffer bbi = ByteBuffer.allocate(EntryFile.INDEX_SIZE);
59 fci.position(byteOffset);
60 if (fci.read(bbi) == -1) {
61 throw new IndexOutOfBoundsException("Specified index is out of range");
62 }
63 bbi.flip();
64 long dataOffset = bbi.getLong();
65 bbi.rewind();
66 long dataOffsetNext;
67 if (fci.read(bbi) == -1) {
68 dataOffsetNext = fcd.size();
69 } else {
70 bbi.flip();
71 dataOffsetNext = bbi.getLong();
72 }
73 int dataSize = (int) (dataOffsetNext - dataOffset);
74
75 // Get the serialized object data in a byte array.
76 byte[] se = new byte[dataSize];
77 fcd.position(dataOffset);
78 fcd.read(ByteBuffer.wrap(se));
79
80 // Deserialize the byte array into an instantiated object.
81 return Entry.deserialize(se);
82 }
83}
Use of the code
The code above can be used as follows:
1import java.io.FileNotFoundException;
2import java.io.IOException;
3import java.util.logging.Level;
4import java.util.logging.Logger;
5
6public class Main {
7
8 public static void main(String args[]) {
9
10 EntryFile entryFile = null;
11 try {
12 entryFile = new EntryFile("entries");
13 for (int i = 0; i < 100; i++) {
14 Entry entry = new Entry("name[" + i + "]", "message[" + i + "]");
15 entryFile.appendEntry(entry);
16 }
17 for (int i = 99; i >= 0; i--) {
18 Entry entry = entryFile.readEntry(i);
19 System.out.println("Entry["+i+"]");
20 System.out.println("\\ttimestamp=" + entry.getTimestamp().toString());
21 System.out.println("\\tusername=" + entry.getUsername());
22 System.out.println("\\tmessage=" + entry.getMessage());
23 }
24 entryFile.close();
25 } catch (FileNotFoundException ex) {
26 Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
27 } catch (IOException ex) {
28 Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
29 } catch (ClassNotFoundException ex) {
30 Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
31 }
32 }
33}
This code creates, serializes and writes 100 objects to file and when done, reads back, deserializes and prints the 100 retrieved objects to standard output, in reverse order.
Advantages of serializing via a byte array
Serializing objects via a byte array instead of directly to file allows for more control over the process. One can add extra information to each stored object, e.g. a length and a checksum. A length is useful when the index file gets corrupted and needs to be regenerated. A checksum can be used to verify the validity of the stored serialized object.