Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Web File on Deno #28

Closed
wants to merge 3 commits into from
Closed

Conversation

KnorpelSenf
Copy link
Member

Allows File objects to be passed to InputFile (on Deno only). The file will be streamed via file.stream(). The filename will be taken from the supplied file object. Retrying API calls is possible. This change is non-breaking.

@wojpawlik is this possible to port to Node, preferably without raising the minimum Node version?
If no, this will simply be merged, and Node users will not get support for web files.

Copy link
Contributor

@wojpawlik wojpawlik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Node.js support is tricky.

Node 16 and fetch-blob@2 have Blob, but no File. And simple instanceof Blob won't catch all Blob implementations.

I was thinking in terms of allowing File instead of InputFile, and adding name getter and stream method to InputFile to make it File-like.

deno/src/platform.ts Outdated Show resolved Hide resolved
deno/src/platform.ts Outdated Show resolved Hide resolved
@KnorpelSenf
Copy link
Member Author

I was thinking in terms of allowing File instead of InputFile, and adding name getter and stream method to InputFile to make it File-like.

InputFile objects are only used with grammY, so there's no need for them to be File-like.

Also, it is a goal to be explicit about the file data that are passed to grammY methods, which is achieved by always relying on the InputFile constructor and nothing else. Permitting File objects contradicts this.

@KnorpelSenf KnorpelSenf marked this pull request as ready for review August 15, 2021 21:02
@KnorpelSenf
Copy link
Member Author

If this PR can be merged as-is, it will land in the 1.3 release, which is imminent.

@wojpawlik
Copy link
Contributor

I'd wait with this. Deno has no fileFromPath yet to take advantage of this. And I'll soonish tackle similar problems in https://github.com/telegraf/client and share my thoughts here.

@KnorpelSenf KnorpelSenf marked this pull request as draft August 16, 2021 08:56
Comment on lines +51 to +55
private [internalInputFileData]: ConstructorParameters<typeof InputFile>[0]
public get [inputFileData]() {
const file = this[internalInputFileData]
return file instanceof Blob ? file.stream() : file
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know what I dislike about this change. This logic should be moved into Client.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

InputFile is semantically part of the client. The two cannot be separated. It just has to reside in a different file for cross-platform reasons.

In a way, the InputFile encapsulates the runtime-specific part of the client. Given that calling stream() only works on Deno, it needs to be part of InputFile.

That being said, I know this way to divide the code is mediocre at best. Maybe d2n will improve the situation.

Copy link
Contributor

@wojpawlik wojpawlik Sep 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

d2n should be able to shim Blob, so that it's possible to if (Blob && foo instanceof Blob) anywhere.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Node 16.7.0 added Blob::stream()

@@ -46,7 +48,11 @@ export const inputFileData = Symbol('InputFile data')
* Reference](https://core.telegram.org/bots/api#inputfile).
*/
export class InputFile {
public readonly [inputFileData]: ConstructorParameters<typeof InputFile>[0]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For 2.0, InputFile should accept Blob instead of path: string (nodejs/node#37340), and expose just name: string and a .stream() method.

Copy link
Member Author

@KnorpelSenf KnorpelSenf Sep 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds like a good idea, that would simply the abstraction to the client 👍🏻

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Further, you could turn InputFile into an interface, and provide class InputFileAsyncIterable instead, perhaps some others. Zero type-switching in your code!

You could still easily detect InputFiles with

function isInputFile(value: any): value is InputFile {
  return value != null && typeof value.stream === 'function'
}

Note that web File already satisfies

interface InputFile {
  readonly name: string
  readonly stream: () => ReadableStream
}

Copy link
Member Author

@KnorpelSenf KnorpelSenf Sep 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, the goal is to be explicit about file uploads by constructing an InputFile object. When bot code is read, it should be immediately obvious that a file upload will be performed. Permitting lots of things in the API directly contradicts this because it allows to circumvent the InputFile constructor.

That being said, I'm open to change completely how InputFile and client work together if there is a better solution.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd argue that new upload.Buffer(uint8array, name) or new upload.Iterable(streamOrIterable, name) or await Deno.getFile(path) is more explicit than new InputFile(oneOfFourTypes).

I don't think it makes sense to ever merge this PR. Better open a fresh one when working on 2.0.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On 2nd thought, this could be made mergable:

  1. Switch to deno2node,
  2. Shim Blob,
  3. Move Blob detection to Client.

But there doesn't seem to be any demand.

@KnorpelSenf KnorpelSenf deleted the web-file-support branch September 12, 2021 14:34
@jimmywarting
Copy link

Node 16 and fetch-blob@2 have Blob, but no File. And simple instanceof Blob won't catch all Blob implementations.

fyi, fetch-blob@3 have a File class, it also has a method to get a Blob/File wrapper out of a file path, so it dose not have to have everything in the memory.

Also it has a whatwg compatible .stream() method

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants