使用 Spring Data Embedded Mongo 在 Mongo db 中导入 JSON 文件

Question

我正在尝试编写一些与需要从 MongoDB 提取数据的方法相关的集成测试。具体来说，我使用的是Spring Data项目给出的Embedded Mongo。内嵌的mongo明明是Flapdoodle提供的。

我需要将一些 json 文件导入 Embedded Mongo。我查看了 flapdoodle 提供的测试，但我无法理解它们如何与 Spring Data + Spring Boot 提供的 magic 集成。

谁能发布一些澄清的sn-ps？

Answer 1

您可以创建一个在每次测试之前和之后运行的 junit 规则 (ExternalResource)。查看MongoEmbeddedRule 类以了解实现细节。

集成测试：

@RunWith(SpringRunner.class)
@SpringBootTest(webEnvironment = RANDOM_PORT)
public abstract class TestRunner {

    @Autowired
    protected MongoTemplate mongoTemplate;

    @Rule
    public MongoEmbeddedRule mongoEmbeddedRule = new MongoEmbeddedRule(this);

外部资源规则：

public class MongoEmbeddedRule extends ExternalResource {

    private final Object testClassInstance;
    private final Map<String, Path> mongoCollectionDataPaths;
    private final String fieldName;
    private final String getterName;

    public MongoEmbeddedRule(final Object testClassInstance) {
        this(testClassInstance, "mongoTemplate", "getMongoTemplate");
    }

    protected MongoEmbeddedRule(final Object testClassInstance, final String fieldName, final String getterName) {
        this.fieldName = fieldName;
        this.getterName = getterName;
        this.testClassInstance = testClassInstance;
        this.mongoCollectionDataPaths = mongoExtendedJsonFilesLookup();
    }

    @Override
    protected void before() {
        dropCollections();
        createAndPopulateCollections();
    }

    @Override
    protected void after() {
    }

    protected Set<String> getMongoCollectionNames() {
        return mongoCollectionDataPaths.keySet();
    }

    public void dropCollections() {
        getMongoCollectionNames().forEach(collectionName -> getMongoTemplate().dropCollection(collectionName));
    }

    protected void createAndPopulateCollections() {
        mongoCollectionDataPaths.forEach((key, value) -> insertDocumentsFromMongoExtendedJsonFile(value, key));
    }

    protected MongoTemplate getMongoTemplate() {
        try {
            Object value = ReflectionTestUtils.getField(testClassInstance, fieldName);
            if (value instanceof MongoTemplate) {
                return (MongoTemplate) value;
            }
            value = ReflectionTestUtils.invokeGetterMethod(testClassInstance, getterName);
            if (value instanceof MongoTemplate) {
                return (MongoTemplate) value;
            }
        } catch (final IllegalArgumentException e) {
            // throw exception with dedicated message at the end
        }
        throw new IllegalArgumentException(
                String.format(
                        "%s expects either field '%s' or method '%s' in order to access the required MongoTemmplate",
                        this.getClass().getSimpleName(), fieldName, getterName));
    }

    private Map<String, Path> mongoExtendedJsonFilesLookup() {
        Map<String, Path> collections = new HashMap<>();
        try {
            Files.walk(Paths.get("src","test","resources","mongo"))
                    .filter(Files::isRegularFile)
                    .forEach(filePath -> collections.put(
                            filePath.getFileName().toString().replace(".json", ""),
                            filePath));
        } catch (IOException e) {
            e.printStackTrace();
        }
        return collections;
    }

    private void insertDocumentsFromMongoExtendedJsonFile(Path path, String collectionName) {
        try {
            List<Document> documents = new ArrayList<>();
            Files.readAllLines(path).forEach(l -> documents.add(Document.parse(l)));
            getMongoTemplate().getCollection(collectionName).insertMany(documents);
            System.out.println(documents.size() + " documents loaded for " + collectionName + " collection.");
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

json 文件 (names.json) 与MongoDB Extended JSON，其中每个文档都在一行中，集合名称是不带扩展名的文件名。

{ "_id" : ObjectId("594d324d5b49b78da8ce2f28"), "someId" : NumberLong(1), "name" : "Some Name 1", "lastModified" : ISODate("1970-01-01T00:00:00Z")}
{ "_id" : ObjectId("594d324d5b49b78da8ce2f29"), "someId" : NumberLong(2), "name" : "Some Name 2", "lastModified" : ISODate("1970-01-01T00:00:00Z")}

Answer 2

您可以查看“flapdoodle”提供的以下测试类。测试展示了如何导入包含集合数据集的 JSON 文件： MongoImportExecutableTest.java

理论上，您也可以导入数据库的整个转储。 （使用 MongoDB 恢复）： MongoRestoreExecutableTest.java

Answer 3

您可以创建一个抽象类并设置逻辑来启动 mongod 和 mongoimport 进程。

AbstractMongoDBTest.java

public abstract class AbstractMongoDBTest {

private MongodProcess mongodProcess;
private MongoImportProcess mongoImportProcess;
private MongoTemplate mongoTemplate;

void setup(String dbName, String collection, String jsonFile) throws Exception {
    String ip = "localhost";
    int port = 12345;

    IMongodConfig mongodConfig = new MongodConfigBuilder().version(Version.Main.PRODUCTION)
            .net(new Net(ip, port, Network.localhostIsIPv6()))
            .build();

    MongodStarter starter = MongodStarter.getDefaultInstance();
    MongodExecutable mongodExecutable = starter.prepare(mongodConfig);

    File dataFile = new File(Thread.currentThread().getContextClassLoader().getResource(jsonFile).getFile());
    MongoImportExecutable mongoImportExecutable = mongoImportExecutable(port, dbName,
            collection, dataFile.getAbsolutePath()
            , true, true, true);

    mongodProcess = mongodExecutable.start();
    mongoImportProcess = mongoImportExecutable.start();

    mongoTemplate = new MongoTemplate(new MongoClient(ip, port), dbName);
}

private MongoImportExecutable mongoImportExecutable(int port, String dbName, String collection, String jsonFile,
                                                    Boolean jsonArray, Boolean upsert, Boolean drop) throws
        IOException {
    IMongoImportConfig mongoImportConfig = new MongoImportConfigBuilder()
            .version(Version.Main.PRODUCTION)
            .net(new Net(port, Network.localhostIsIPv6()))
            .db(dbName)
            .collection(collection)
            .upsert(upsert)
            .dropCollection(drop)
            .jsonArray(jsonArray)
            .importFile(jsonFile)
            .build();

    return MongoImportStarter.getDefaultInstance().prepare(mongoImportConfig);
}

@AfterEach
void clean() {
    mongoImportProcess.stop();
    mongodProcess.stop();
}

public MongoTemplate getMongoTemplate(){
    return mongoTemplate;
}

}

YourTestClass.java

public class YourTestClass extends AbstractMongoDBTest{


@BeforeEach
void setup() throws Exception {
    super.setup("db", "collection", "jsonfile");
}

@Test
void test() throws Exception {

}

}