normalize parser struct for all file types

[timothycarambat]

No description provided.

[review-agent-prime]

Here are some suggestions to improve the code:

1. In as_docx.py and as_text.py, you have hardcoded the docAuthor and description fields as 'Unknown'. It would be better to extract these details from the document metadata if possible. If the metadata is not available, then default to 'Unknown'.

'docAuthor': get_author(fullpath) if get_author(fullpath) else 'Unknown',
'description': get_description(fullpath) if get_description(fullpath) else 'Unknown',

2. In as_mbox.py, you have removed the sender, recipient, subject, and date_sent fields. If these fields are not necessary for your use case, that's fine. However, if you need this information later, consider keeping these fields.

"sender": message["From"],
"recipient": message["To"],
"subject": subject,
"date_sent": date_sent,

3. In as_text.py, you have added a docSource field. This is a good addition, but it would be better to make the source more descriptive, such as including the file path.

'docSource': f"a text file uploaded by the user from {fullpath}",

4. In all files, you are calculating the word count using len(content). This will give you the character count, not the word count. To get the word count, you should split the content by spaces.

'wordCount': len(content.split()),

5. In all files, you are using the file_creation_time(fullpath) function to get the published date. If the file creation time is not the same as the published date, consider using a different method to get the correct date.