API - Clarify Pitch vs Normal Loader

[pksjce]

From @gaearon on July 15, 2014 8:10

I'd submit a PR but honestly, having written two loaders, I'm not sure I really understand what a pitch loader is.

Is my understanding correct here:

• normal loaders always transform the source and thus rely on previous loader's output
• pitch loaders don't depend on the source itself and are happy with being given just a module name (that they can require), thus they are cacheable even if module's content changes
• everything that can be implemented as a pitch loader, can also be implemented as a normal loader, but pitch loaders are faster (cacheable with no regards to module output, may be run in parallel, etc)

?

Copied from original issue: webpack/webpack#360

[pksjce]

From @jhnns on July 15, 2014 10:2

The loader api is pretty confusing imho. Maybe it should be streamlined someday? 😄

[pksjce]

From @sokra on July 15, 2014 13:12

@gaearon correct.

It's like the two phases of event bubbling...

a!b!c!resource

• pitch a
  • pitch b
    • pitch c
      • read file resource (adds resource to dependencies)
    • run c
  • run b
• run a

When a loader return something in the pitch phase the process continues with the normal phase of the next loader... Example:

• pitch a
  • pitch b (returns something)
• run a

[pksjce]

From @jhnns on July 15, 2014 13:57

What's the use-case for this behavior?

[pksjce]

From @dashed on July 15, 2014 18:14

@jhnns If I recall, better documentation is one of the objectives of webpack 1.0, if not 1.x. So it's good to address it asap.

Perhaps if we create an issue to consolidate any and all terminology used in the docs/source code?

I've yet to thoroughly use all the loader concepts. From @sokra's clarification, it seems 'pitch loader' is something of a 'staging' component of a loader.

I looked up the definition of the word pitch in the context of the loader:

verb (used with object) to put, set, or plant in a fixed or definite place or position.

Since it's not obvious that loaders have different ways of being executed. Thus, let's rename 'pitch loader' to something sensible: mark stage.

Then we have different stages as part of a loader:

1. mark stage
2. execution/run stage

From what I read from the docs, I'm unsure if there are any other stages.

@sokra Would defining/clarifying loader stages in docs make sense? Although, it may not be explicitly defined in webpack source code.

[pksjce]

From @sokra on July 15, 2014 19:0

@jhnns You can/should use it if you don't want to modify the source code, but just insert a module in front for the module. i. e. the bundle-loader uses this.

@dashed yes, pitch is propably a bad word here... maybe capture is the better word (similar to the capturing phase of DOM events). I think we can rename it if we can agree on a name...

[pksjce]

From @jhnns on July 15, 2014 19:53

capture and bubbling phase seems pretty straight-forward - at least to me. But it might be confusing since loaders are not DOM objects 😉. What do the others say?

[pksjce]

From @gaearon on July 15, 2014 20:40

I get the capture/bubbling metaphor but I'm not sure I understand the use cases yet. It would help to have a page explaining how several different loaders work together and why having these different phases is useful.

[pksjce]

From @dashed on July 15, 2014 22:37

I think, overall, the loader docs needs some overhaul.

Concepts and ideas are pretty much introduced as examples, and assumes prior knowledge: http://webpack.github.io/docs/loaders.html#examples

The loader docs should have sub-sections as follows:

• execution flow: async and sync
• loader stages/phases: pitch and run (normal loader)
• content feeding for run stage: utf-8 string and buffer

github blame doesn't reveal the original reason for pitch loaders: webpack/core@ea1e218 It's probably refactored code.

---

I agree with @jhnns that, unless loaders is in some way very related to 'event dom bubbling', it's best to refrain from borrowing terminology from there to avoid any further confusion.

It seems from @sokra's explanation of pitch loaders, a 'break phase/stage' is probably a better terminology, akin to break; keyword for loops.

[pksjce]

From @gaearon on October 28, 2014 9:34

I feel loaders are most naturally explained as middleware for require.

[pksjce]

From @adjavaherian on July 30, 2015 17:46

I've been writing a plugin and a loader and its nearly impossible to understand how to do this given the current documentation, that's probably why there around 10x as many browserify plugins in comparison. It took me a day to write a meaningful browserify plugin and a week to do the same in webpack, and I'm still stuck. I've literally been writing the plugin docs myself as I find more information about them. I think webpack does a few things really well, The plugin, compiler and the compilation and chunks are all really fantastic and well thought out, but its a total gray box. There's little to no documentation on the interfaces. Those APIs should be well documented with examples.

The loaders have pre-loaders, pitching loaders, post loaders, and who knows what else? And its the same story regarding documentation, but perhaps a little less complicated to suss out from reading the source of other loaders.

But my biggest frustrations so far are not having an inkling of an idea of how to simply inject a module into the compilation. Or for example, if you instantiate webpack with the node api, how do you get access to the compilation object or simply return an object as opposed to writing json to the file system? This is the kind of thing that should be easy, no?

[pksjce]

From @mik01aj on November 12, 2015 14:24

+1 +1 +1. Have the same experience. I would love to read some better documentation about it.

[pksjce]

From @IngwiePhoenix on November 12, 2015 14:53

Yup. The docs are usually what people actually leads into mislead information. I have been doing a lot of supportive work on Gitter lately - and i would find myself grabbing paragraphs of docs and link these specific paragraphs, and the uses would be like "Oh! I totally oversaw that."

Loaders are generally OK with their explanation, but the deeper-down details seem lacking. Especially for devs.

• When should I make a pre-loader?
• When should I make a post-loader?
• What is a pitch-loader and how can they be used?
• How do I combine the use of a Plugin and Loader?

For instance, say you have your new scripting language that has a runtime. You could prepend a require statement to all the sources through a loader, OR provide an entry through the ProviderPlugin. Or, something else entirely. It's unclear.

I really like WebPack and I am totally set on it for many codeworks to come. But I really wish it would clarify a few terms and things.

Finally I had seen the "How to write a plugin" page having gotten an update, I am grateful for that. But there are still many common usecases that people usually have to dig up from other sources... or, other source codes entirely.

Just leaving these two cents here, in hope that we may be able to get better webpack docs.

I would gladly help. I mean, I have a good general understanding of it now, and even have had read the source at a few places. So all I can do is suggest and think, for now.

[pksjce]

From @jhnns on November 12, 2015 14:58

I totally agree 👍

[pksjce]

From @amireh on March 30, 2016 10:59

Sorry to resurrect an old thread. Sokra's comment in this thread on how the pitch phase works is invaluable and I believe should be added to the docs!

I'm trying to add support for pitching loaders in happypack (see amireh/happypack#15) and now I realize after reading this thread and receiving some comments that there are also preLoaders / postLoaders in the loader application process.

Is there any chance we can formally specify the loader application process?

My second question is: by chance I happened to land upon https://github.com/webpack/loader-runner which is basically another version of what I had to write in https://github.com/amireh/happypack/blob/pitch-loaders/lib/applyLoaders.js to apply the loaders outside of a webpack compilation context.

Can we please consolidate the efforts?

It would be hard to maintain something like happypack when it relies on webpack's implementation details that could change between versions as I need to stay on top of it. However, the implementation in loader-runner makes assumptions that wouldn't work for happypack (like all the APIs that need to run within the compilation context).