Skip to content

Scan for malformed CSV files to avoid SIS Upload Error #104

Open

Description

Frequently a CSV files comes along that causes a "SIS Upload Error". One often reason for this the CSV files contain unescaped quotes. Something along the lines of

01234567,testuser,,"Nickname"","Firstname",testuser@umich.edu,active

The problem is that these CSV files are contained within a zip file that we don't do any processing to. To fix this we'd have to uncompress the zip files, scan for malformed content and fix it and re-compress. The correct fix to escape double quotes like this is to put an extra double quote in front of it.

Currently fixing this is a manual process. It was believed that they'd try to fix this upstream (TDX 231033) but feels like this has been a problem for a few months so might be worth looking at, or at least doing processing for in the new process of the uploader if this was rewritten. The upstream fix looks like it was just to prevent these characters from being entered, so that still seems like we might see problems with the downstream data.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions