Support locating non-page js/jsx files with page files

[zenflow]

Original title: Support having non-page js/jsx files in pages dir

This is an often requested feature. See issues #3508 #1914 #1689 #1545 #988 and there are likely more. edit: Also #3183.

This issue was fixed in PR #3195 but somewhere along the way the pagesGlobPattern config became broken, and was cleaned out in PR #3578.

This feature would allow us to place files that support specific pages, with the pages that they support. These "support" files could be tests, components, utilities, and more. This would improve the application's file organization, especially for large projects.

[zenflow]

For clarification

This is the folder structure we want to avoid, especially if it is to scale to have more than 3 pages in a hierarchy more than 1 deep:

• components/
  • chat/
  • timeline/
  • (... index components)
• helpers/
  • chat/
  • timeline/
  • (... index helpers)
• pages/
  • chat.js
  • timeline.js
  • index.js
• tests/
  • chat/
  • timeline/
    (... index tests)

This would be more maintainable at scale, since we don't duplicate the (root->(chat, timeline)) structure for each file type:

• pages/
  • __tests__/
  • lib/
    • components/
    • helpers/
  • chat/
    • __tests__/
    • lib/
      • components/
      • helpers/
    • index.js
  • timeline/
    • __tests__/
    • lib/
      • components/
      • helpers/
    • index.js
  • index.js

[zenflow]

Don't see any harm in adding this 👍 I do want to keep this as 'internal' (undocumented) cause it's not a practice we endorse. @timneutkens in PR #3195

@timneutkens Why would you not endorse this practice?

[zenflow]

I see two solutions:

1. Re-implement pagesGlobPattern config, but this time:
   • remove the 'pages/' bit from the default, and just prefix it before using, since this cannot be changed (pages must be in the pages/ dir, otherwise this solution becomes more complicated)
   • add tests to make sure there is not a regression
2. Take a convention-over-configuration approach, and make all lib/, __tests__/, and __mocks__/ folders ignored. __test__/ and __mock__/ (singular) too

[zenflow]

@timneutkens @sergiodxa Would you be open to a PR for either of the above solutions? Or perhaps a different solution?

[moaxaca]

This really is a deal breaker for people designing large more enterprise applications.

Some notable solutions that allow similar functionality:
https://www.gatsbyjs.org/ Static Generation
https://github.com/jaredpalmer/after.js SSR - React Router

If I had the community support, I would consider making my fork something more modularized & flexible then pushing it to npm something like flext.

[timneutkens]

You could actually try to do this:

1. Make a directory, eg components
2. Put all the code that you want to put into pages into components
3. Use symlinks to link the actual pages to the pages directory

This gives more granular control over routes that are accessible to the outside world. And is safer and clearer than providing exclude/includes.

but somewhere along the way the pagesGlobPattern config became broken

To be clear, pagesGlobPattern was always broken. It only accounted for next build scenario. And I removed it because it was obsolete because of that.

This really is a deal breaker for people designing large more enterprise applications.

We're continuously adding features/improvements to face bigger scale, as our own app grows (currently over a thousand components), and as the community gives feedback.

[moaxaca]

@timneutkens That's clever, I will give it a shot. Hopefully, the windows guys on my team don't hate me for committing a symlink if it works!

This being said, I would prefer to see a roadmap or a consensus from the maintainers on pivotal features.

[rauchg]

@moaxaca the only reason we're hesitant to support the glob is precisely because we are concerned that at scale it could open critical security concerns.

We basically would be encouraging people to put into pages/ things that are not meant to be routable, that they have to carefully exclude (as opposed to opt into and include explicitly).

For example, a side effect of not excluding __tests__ in the example above would be to execute code that results in denial of service, because it's not meant to be run in the context of a HTTP request.

I don't consider it "enterprise ready" to go back on what's otherwise a very clear and secure pattern, but we're always listening to the community for different approaches, in case one is suggested here.

In fact, it suffices to do some light research into the vast literature of PHP vulnerabilities to find many that fall under "I exposed a .php file to the end user that was not meant to be world-accessible".

[zenflow]

@timneutkens I am a Windows guy (don't judge) so the symlink approach will not work for me, and besides, there should be a portable solution. But thank you very much for the suggestion!

[zenflow]

@rauchg

but we're always listening to the community for different approaches, in case one is suggested here.

What about the second solution that I proposed above?

Take a convention-over-configuration approach, and make all lib/, __tests__/, and __mocks__/ folders ignored. __test__/ and __mock__/ (singular) too

It seems that this would be less error-prone; users would not have to carefully exclude the files (i.e. write a glob pattern), they just have to make sure that they don't misspell lib or __tests__ or __test__.

This would also be a very clear pattern I think, no?

[zenflow]

If this is still too error-prone, maybe we could still work with this idea, and extend it with something like "print a warning if a dir like lbi is detected", or something else

[gihrig]

I worked around this issue as follows:

Think of ./pages as "routes", move existing pages content to another directory.

Create ./containers directory
Move contents of ./pages into ./containers
Create 'link files' in pages. e.g. ./pages/index.js:

import Home from '../containers/Home'

export default Home

This allows for any complex folder structure in ./pages for matching complex url's and does not risk unwanted exposure of content that should not be public.

[zenflow]

Beautiful solution @gihrig! 🎉 This solves the problem!

Though I still would prefer not to have to write modules like:

import Foo from '../../../../../path/to/deeply/nested/container/Foo'

export default Foo

and if it was approved I would still make a PR implementing what I proposed above.

[rauchg]

By the way, I also think we do need a solution to not having to write so many ../../, but I think that can be done with babel? It's nice to be able to register a "root", and be able to use it anywhere like so:

import "containers/…"

That's also generally useful throughout the codebase, so maybe @gihrig's solution has to be extended with some babel plugin in case you have too many nested paths.

I believe @hanford is using such a plugin successfully already.

tl;DR: @gihrig's solution is sufficient. It can be combined with the "root" import solution which is orthogonal and generally useful beyond pages/

[hanford]

@rauchg @zenflow - We're using babel-plugin-module-resolver at Eaze, and I'm a huge fan