Skip to content

Low level MIME sniffing #49843

Closed as not planned
Closed as not planned
Low level MIME sniffing#49843
Labels
feature requestIssues requesting new Node.js features.staleIssues and PRs marked stale due to inactivity and scheduled for automatic closure.

Description

@jimmywarting
jimmywarting
opened on Sep 24, 2023

What is the problem this feature will solve?

This issue came up during a discussion related to fs.openAsBlob, where blob.type currently returns an empty string. I must agree that it would be nice if Node.js could automatically determine the MIME type of a given entity, whether through extension lookup or by inspecting "magic numbers".

Of course, there might be some cases where certain file extensions collide, but I believe it would be best to adhere to the IANA standards. After all, all browsers are capable of deducing certain information. Here's a straightforward example:

var root = await navigator.storage.getDirectory()
var fileHandle = await root.getFileHandle('image.png', { create: true })
var file = await fileHandle.getFile()
console.log(file.type) // Returns 'image/png'

I'm not sure about the exact mechanism behind it, but it simply works somehow.

I've come across some useful resources that might help address this issue:

These resources could provide valuable insights and potential solutions for improving the MIME type detection in Node.js.

What is the feature you are proposing to solve the problem?

I'm unsure about the API's potential appearance, but it would be beneficial to explore various methods for detecting it. I'm also uncertain whether it should return an array of potential matches along with their respective scores and probabilities.

For instance:

x.identifyType('txt') // Returns 'text/plain'
x.identifyExtension('text/plain') // Returns 'txt'
x.sniffTypeFrom(blob || arrayBuffer || arrayBufferView) // Infers 'text/plain'
x.sniffExtensionFrom(blob || arrayBuffer || arrayBufferView) // Infers 'txt'
x.standardize('text/javascript1.2') // Converts to 'text/javascript' following the specification at https://mimesniff.spec.whatwg.org/#ref-for-javascript-mime-type
x.detectFormat(anything) // Returns { extension: 'txt', type: 'text/plain' }
x.lookupInSystemRegistry(anything) // Searches, for instance, the Windows system registry for installed applications
x.IANA...

Perhaps when searching for a path, it could consider both the file name extension and perform MIME sniffing.

Lastly:

blob = await openAsBlob('./image.png')
blob.type // Returns 'image/type'

Please note that the exact method names and functionality would depend on the implementation details of the API.

I think this would serve the community to have this built right in as it's something that is very commonly used.

Activity

  1. jimmywarting
    added
    feature requestIssues requesting new Node.js features.
    on Sep 24, 2023
  2. github-project-automation
    added this to Node.js feature requestson Sep 24, 2023
  3. github-project-automation
    moved this to Pending Triage in Node.js feature requestson Sep 24, 2023

  4. bnoordhuis commented on Sep 25, 2023

    @bnoordhuis
    bnoordhuis
    on Sep 25, 2023
    Member

    whether through extension lookup or by inspecting "magic numbers"

    Both have problems. Just looking at the extension is prone to false positives (e.g. GIF file with a .jpg extension), while sniffing the first few bytes is problematic with non-files like fifos (can't seek, can't rewind.)

    Even determining if a file is text/plain is challenging if you don't know the encoding. A file consisting of the octets 00 00 00 20 is text/plain only when you know it's UTF32-BE, otherwise it's application/octet-stream.

    https://mimesniff.spec.whatwg.org/ seems to be aimed at HTTP, not file systems, so I don't think it's of much use. I feel node shouldn't be in the business of making educated guesses, not when everyone else is making different educated guesses.

  5. anonrig commented on Sep 25, 2023

    @anonrig
    anonrig
    on Sep 25, 2023
    Member

    For future reference, we have a fast mimesniff parser at Ada - https://github.com/ada-url/mimesniff

  6. Uzlopak commented on Oct 16, 2023

    @Uzlopak
    Uzlopak
    on Oct 16, 2023
    Contributor

    A million years ago i programmed this

    https://github.com/parallax/jsPDF/blob/5d09af9135a2fe049c7d3c8b95df280d22e4a6db/src/modules/addimage.js#L51

    Is this something we should put in a transform stream?

  7. github-actions commented on Apr 14, 2024

    @github-actions
    github-actionsbot
    on Apr 14, 2024 – with GitHub Actions
    Contributor

    There has been no activity on this feature request for 5 months. To help maintain relevant open issues, please add the never-stale Issues and PRs exempt from automated stale handling. label or close this issue if it should be closed. If not, the issue will be automatically closed 6 months after the last non-automated comment.
    For more information on how the project manages feature requests, please consult the feature request management document.

  8. github-actions
    added
    staleIssues and PRs marked stale due to inactivity and scheduled for automatic closure.
    on Apr 14, 2024

  9. github-actions commented on May 15, 2024

    @github-actions
    github-actionsbot
    on May 15, 2024 – with GitHub Actions
    Contributor

    There has been no activity on this feature request and it is being closed. If you feel closing this issue is not the right thing to do, please leave a comment.

    For more information on how the project manages feature requests, please consult the feature request management document.

  10. github-actions
    closed this as not plannedon May 15, 2024
  11. avivkeller
    removed this from Node.js feature requestson Jun 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    feature requestIssues requesting new Node.js features.staleIssues and PRs marked stale due to inactivity and scheduled for automatic closure.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions