【发布时间】：2014-01-06 11:42:12
【问题描述】：
背景
我正在尝试以这样一种方式编写 Kryo 反序列化：如果对象数组包含 一些 对象（由于代码更改）无法反序列化，那么数组中的那些引用将变为 null 而不是抛出异常；允许回收对象的其余部分。我以前一直在使用 Java 的内置序列化，并且在其中我能够通过在数组中的每个项目之间写入一个“已知良好”的整数来实现这一点，然后在发生错误时在流中查找它以找到开始下一个对象。这是问题Deserializing an array that contains some non-deserializable objects (salvaging the deserializable parts)的详细信息。
出于效率的原因，我现在已经转移到 Kryo 序列化，并尝试重新创建这种方法，但是在 Kryo 中，这种错误恢复似乎可以工作一次，但之后就无法正确恢复。
我尝试过的
我试图在序列化期间在Apples 数组中的每个对象之间写入一个已知的良好整数（END_OF_APPLE_MAGIC）。在反序列化过程中，当发现无法反序列化的BadApple 时，将其替换为ErrorApple（类比越来越弱），并搜索END_OF_APPLE_MAGIC 以查找下一个苹果的位置。如果数组中有一个 BadApple 并且 BadApple 不是第一个条目，则此方法有效。但是如果数组中有多个BadApple 或第一个Apple 是BadApple，则会以各种方式失败（请参阅详细分析）
public class AppleHolder implements Serializable,KryoSerializable{
static int END_OF_APPLE_MAGIC=1234467895; //if this just "turns up" in the stream we will have a bad day; however this is only the case in recovery mode, so is an acceptable risk
int numberOfApples=6;
Apple[] apples=new Apple[numberOfApples];
double otherData=15;
//these are just for debug
int dividers=0; //counts number of times END_OF_APPLE_MAGIC is found
int problems=0; //counts number of times an apple fails to load
int badIntegers=0; //counts number of times END_OF_APPLE_MAGIC is looked for and a different Integer is found (I have never seen this happen)
public AppleHolder(){
Apple goodApple=new Apple("GoodApple","tastyGood");
BadApple badApple=new BadApple("BadApple","untastyBad");
apples[0]=goodApple;
apples[1]=badApple;
apples[2]=goodApple;
apples[3]=goodApple; // multiple references to same object intentional
apples[4]=goodApple;
apples[5]=goodApple;
}
public void write (Kryo kryo, Output output) {
for(int i=0;i<apples.length;i++){
//kryo.writeObject(output, apples[i]);
kryo.writeClassAndObject(output, apples[i]);
kryo.writeClassAndObject(output, END_OF_APPLE_MAGIC);
}
kryo.writeObject(output,otherData);
}
public void read (Kryo kryo, Input input) {
try{
apples =new Apple[numberOfApples];
for(int i=0;i<apples.length;i++){
try{
Object ob=kryo.readClassAndObject(input);
apples[i]=(Apple)ob;
}catch(Exception e){
apples[i]=new ErrorApple();
problems++;
}
//Search for next Apple Boundary (except in recovery mode
//it will be the next entry)
boolean atBoundary=false;
while (atBoundary==false){ //should probably put a limit on this just in case
try{
int appleMagic =(Integer)kryo.readClassAndObject(input);
if (appleMagic == END_OF_APPLE_MAGIC){
atBoundary=true;
dividers++;
}else{
badIntegers++;
}
}catch(Exception e){
//painful byte reading mode only entered in recovery mode; under good
//situations it does not represent an efficiency problem
input.skip(1); //consume byte of bad input
//Where buffer underflow exceptions occur they occur here
}
}
}
otherData = kryo.readObject(input, Double.class);
}catch(Exception e){
//something when wrong (always a Buffer underflow so far), print what we have
for(int i=0;i<apples.length;i++){
System.out.println(apples[i]);
}
throw e;
}
}
public static void main(String[] args)
throws Exception {
/*
* (1) First run serialize()
* (2) Rename/delete badApple such that it cannot be found for deserialization
* (3) Run deSerialize(()
*/
serialize();
//deSerialize();
}
public static void serialize() throws Exception{
AppleHolder testWrite = new AppleHolder();
/*FileOutputStream fos = new FileOutputStream("testfile");
ObjectOutputStream oos = new ObjectOutputStream(fos);
oos.writeObject(testWrite);
oos.flush();
oos.close();
*/
Kryo kryo = new Kryo();
Output output = new Output(new FileOutputStream("testfile"));
kryo.writeObject(output, testWrite);
output.close();
}
public static void deSerialize() throws Exception{
/*AppleHolder testRead;
FileInputStream fis = new FileInputStream("testfile");
ObjectInputStream ois = new ObjectInputStream(fis);
testRead = (AppleHolder) ois.readObject();
ois.close();
*/
Kryo kryo = new Kryo();
Input input = new Input(new FileInputStream("testfile"));
AppleHolder testRead = kryo.readObject(input, AppleHolder.class);
input.close();
for(int i=0;i<testRead.apples.length;i++){
System.out.println(testRead.apples[i]);
}
System.out.println("otherData: " + testRead.otherData);
}
}
public class Apple implements Serializable {
private String propertyOne;
private String propertyTwo;
public Apple(){}
public Apple(String propertyOne, String propertyTwo) {
this.propertyOne = propertyOne;
this.propertyTwo = propertyTwo;
validate();
}
private void writeObject(ObjectOutputStream o)
throws IOException {
o.writeObject(propertyOne);
o.writeObject(propertyTwo);
}
private void readObject(ObjectInputStream o)
throws IOException, ClassNotFoundException {
propertyOne = (String) o.readObject();
propertyTwo = (String) o.readObject();
validate();
}
private void validate(){
if(propertyOne == null ||
propertyOne.length() == 0 ||
propertyTwo == null ||
propertyTwo.length() == 0){
throw new IllegalArgumentException();
}
}
public String getPropertyOne() {
return propertyOne;
}
public String getPropertyTwo() {
return propertyTwo;
}
@Override
public String toString() {
return "goodApple";
}
}
public class BadApple extends Apple {
public BadApple(){}
public BadApple(String propertyOne, String propertyTwo) {
super(propertyOne, propertyTwo);
}
@Override
public String toString() {
return "badApple";
}
}
public class ErrorApple extends Apple {
public ErrorApple(){}
public ErrorApple(String propertyOne, String propertyTwo) {
super(propertyOne, propertyTwo);
}
@Override
public String toString() {
return "errorApple";
}
}
问题
如何挽救一个只有部分对象可反序列化的 Kyro 序列化数组？从而为不可反序列化的部分获得一个包含ErrorApple 条目的数组。在我的数组中，我在一个数组中有多个对同一个对象的引用，必须在反序列化过程中保留这些引用。
所以进入序列化我有
[GoodApple]
[GoodApple]
[GoodApple]
[BadApple]
[BadApple]
[GoodApple]
我想要从反序列化中出来（因为 badApple 已经改变并且无法反序列化
[GoodApple]
[GoodApple]
[GoodApple]
[ErrorApple]
[ErrorApple]
[GoodApple]
我希望它提供一个后备方案，在无法实现向后兼容性或对我之前安装的程序的第 3 方修改被删除的情况下
详细分析
本节概述了现有程序失败的方式。
一般
- 数组中第一个位置以外的某个位置的单个
BadApple将正常工作 - 数组中第一个位置的
BadApple将导致下一个Apple正确读取然后ErrorApples从那时起（即使是好的Apples） - 如果有超过 1 个
BadApple，第一个好的Apple在第二个BadApple之后会正确读取但可能会在数组中向前移动，然后从那时起ErrorApples（即使是好的Apples）。将有一个KryoException: Buffer underflow，并且数组末尾可能有空条目。
我使用的输入和输出如下所示：
进出 [好苹果] [好苹果] [好苹果] [好苹果] [坏苹果] [坏苹果] [好苹果] [好苹果] [好苹果] [好苹果] [好苹果] [好苹果] 进出 [坏苹果] [错误苹果] [好苹果] [好苹果] [goodApple] [errorApple] [goodApple] [errorApple] [goodApple] [errorApple] [goodApple] [errorApple] 进出 [好苹果] [好苹果] [坏苹果] [错误苹果] [好苹果] [好苹果] [坏苹果] [错误苹果] [好苹果] [好苹果] [goodApple] [errorApple] KryoException：缓冲区下溢。 （发生在 input.skip(1);) 进出 [好苹果] [好苹果] [好苹果] [好苹果] [坏苹果] [错误苹果] [坏苹果] [错误苹果] [好苹果] [好苹果] [goodApple] [errorApple] KryoException：缓冲区下溢（发生在 input.skip(1);） 进出 [好苹果] [好苹果] [坏苹果] [错误苹果] [坏苹果] [错误苹果] [坏苹果] [好苹果] [goodApple] [errorApple] [goodApple] [空] KryoException：缓冲区下溢。 （发生在 input.skip(1);)【问题讨论】：
标签： java arrays serialization kryo