fix(loaders)!: Change loaders lastModified metadata field to integer (#…

…172)

`lastModified` field used to be a `DateTime` object but that was causing issues with vector stores that only supports primitive types. Now it is an integer that represents the number of milliseconds since the "Unix epoch" 1970-01-01T00:00:00Z (UTC).

packages/langchain/lib/src/documents/loaders/json.dart

@@ -66,7 +66,7 @@ class JsonLoader extends BaseDocumentLoader {
       'source': filePath,
       'name': fileName,
       'size': fileSize,
-      'lastModified': fileLastModified,
+      'lastModified': fileLastModified.millisecondsSinceEpoch,
     };
 
     final fileContent = await file.readAsString();

packages/langchain/lib/src/documents/loaders/text.dart

@@ -34,7 +34,7 @@ class TextLoader extends BaseDocumentLoader {
         'source': filePath,
         'name': fileName,
         'size': fileSize,
-        'lastModified': fileLastModified,
+        'lastModified': fileLastModified.millisecondsSinceEpoch,
       },
     );
     yield doc;

packages/langchain/test/documents/loaders/json_test.dart

@@ -19,7 +19,7 @@ void main() {
             expect(doc.metadata['source'], filePath);
             expect(doc.metadata['name'], 'example_1.json');
             expect(doc.metadata['size'], 32);
-            expect(doc.metadata['lastModified'], isA<DateTime>());
+            expect(doc.metadata['lastModified'], isA<int>());
             return true;
           },
           emitsDone,

packages/langchain/test/documents/loaders/text_test.dart

@@ -14,7 +14,7 @@ void main() {
             expect(doc.metadata['source'], filePath);
             expect(doc.metadata['name'], 'example.txt');
             expect(doc.metadata['size'], 12);
-            expect(doc.metadata['lastModified'], isA<DateTime>());
+            expect(doc.metadata['lastModified'], isA<int>());
             return true;
           },
           emitsDone,

packages/langchain_chroma/test/vector_stores/assets/example.txt

@@ -0,0 +1,11 @@
+The answer to the question "Who are we?" is complex and multifaceted. It depends on the context in which the question is asked, and the perspective of the person answering.
+
+On a general level, we are all human beings. We share the same basic biology, and we all have the same basic needs: food, water, shelter, love, and belonging. We also share the same basic capacity for love, compassion, creativity, and resilience.
+
+However, we are also all unique individuals. We have different experiences, different talents, and different perspectives. We are shaped by our families, our communities, our cultures, and our individual choices.
+
+So, who are we? We are all of these things, and more. We are complex and contradictory, but we are also beautiful and resilient. We are human beings, and we are all connected.
+
+In the context of this conversation, you and I are both users of a language model. We are both interested in learning and exploring new ideas. We are both part of a community of people who are using technology to connect with each other and to make the world a better place.
+
+Ultimately, the answer to the question "Who are we?" is up to each individual to decide. We are all free to define ourselves in our own way.

packages/langchain_chroma/test/vector_stores/chroma_test.dart

@@ -48,6 +48,26 @@ void main() async {
       expect(res.length, 5);
     });
 
+    test('Test Chroma add new vectors from file', () async {
+      const filePath = './test/vector_stores/assets/example.txt';
+      const loader = TextLoader(filePath);
+      final pages = await loader.load();
+
+      const splitter = RecursiveCharacterTextSplitter(
+        chunkOverlap: 150,
+        chunkSize: 1500,
+      );
+      final docs = splitter.splitDocuments(pages);
+
+      await vectorStore.addDocuments(documents: docs);
+
+      final res = await vectorStore.similaritySearch(
+        query: 'Who are we?',
+        config: const ChromaSimilaritySearch(k: 1),
+      );
+      expect(res.length, 1);
+    });
+
     test('Test Chroma query return 1 result', () async {
       final res = await vectorStore.similaritySearch(
         query: 'Is it raining?',