Improve Architectural Design

[sizzlemctwizzle]

I'm essentially just one man (on the backend) trying to create a very large feature-rich site in a short amount of time. I'm mainly focused on functionality so the structure of the code is pretty haphazard. I know there are better ways to structure things but my initial implementation is going to be pretty straight forward with simple abstractions when it makes things easier. When the site is more stable and complete, we should go over the code and try to simplify it with better abstractions.

Note: There are a lot of nested async calls in the code. When a nested call depends on the result of the enclosing function async.series can be used. When the async calls are independent but something needs to be done upon completion of all the calls async.parallel can be used. These functions also make error handling easier.

[jerone]

I agree. Let's see how the first version will looks like and go from there.

You're doing a great job btw.

[sizzlemctwizzle]

You're doing a great job btw.

Thanks.

[nathanathan]

Hi, I have a suggestion for architecture. I'm totally new the the project, so I hope this doesn't come across as intrusive or naive.

I think it would be a good idea separate all the view logic into a client side app and only use node.js to develop a JSON API that serves dynamic data and handles actions. If hosting costs are a concern, you can serve static content from services like gh-pages for free, so moving as much of the application into client-side code as possible minimizes expenses.

The JSON API would have the additional benefit of allowing other sites (or user scripts!) to access your user script database. I think allowing user scripts to access the API could be very useful. For example, I would be interested in developing a user script that opens a code editor on the present webpage that allows users to write, test, and publish scripts for that webpage without ever needing to leave it.

[sizzlemctwizzle]

I think it would be a good idea separate all the view logic into a client side app and only use node.js to develop a JSON API that serves dynamic data and handles actions.

I've done this for another smaller project when I had access to front-end web developer and it worked well. But I'm a server-side developer, and since I'm the primary developer atm, I really don't want to do this right now. Maybe we could do this somewhere down the line if others are interested.

The JSON API would have the additional benefit of allowing other sites (or user scripts!) to access your user script database.

I already planned on adding a JSON API eventually for this exact purpose. I want to make the site really easy to write user scripts for. It makes contributing to the site much easier.

For example, I would be interested in developing a user script that opens a code editor on the present webpage that allows users to write, test, and publish scripts for that webpage without ever needing to leave it.

I thought about creating a patch for Greasemonkey to publish scripts onto the site, but that idea sounds very interesting and useful.

you can serve static content from services like gh-pages for free

Cross-site xhr? Is that even possible with gh-pages? Maybe it could be done by inserting script tags?

I'm totally new the the project, so I hope this doesn't come across as intrusive or naive.

Input from anyone is always welcome. Thanks for the suggestions. Please stick around.

[nathanathan]

I've done this for another smaller project when I had access to front-end web developer and it worked well. But I'm a server-side developer, and since I'm the primary developer atm, I really don't want to do this right now. Maybe we could do this somewhere down the line if others are interested.

I would be interested in pitching in on this. I have a lot of front-end experience.

Cross-site xhr? Is that even possible with gh-pages?

Yes, you just to use set the Access-Control-Allow-Origin header on the API server like this: res.header("Access-Control-Allow-Origin", "*")

It would allow anyone to access the data, but you can add an API key if that turns out to be a problem.

[sizzlemctwizzle]

I would be interested in pitching in on this. I have a lot of front-end experience.

I'm doing this partly for a college course (and partly for fun) so I have deadlines that need to be met (see milestones). I'd like to wait until those first three milestones are met before deciding whether or not to go down this route (hence this issue's milestone).

Yes, you just to use set the Access-Control-Allow-Origin header on the API server

Wow. Didn't realize it was that simple. Thanks for the knowledge. I'll definitely use this when I get around to the API.

Edit: Changed a bunch of stuff as I was thinking.

[nathanathan]

By the way, I have some ideas that I was wondering if it would be possible to eventually dovetail into OpenUserJS.org

The main idea is to create a "command engine" where users can type in natural language queries to trigger scripts. Scripts would be annotated with trigger templates like "Where is {{place}}", so there wouldn't necessarily be any complicated NLP stuff involved.

The other aspect to it is using context annotations so that it's possible to create commands that are only meant to be used after other commands, or, like user scripts, on certain websites.
I've written up a proposal for what I have in mind in greater detail here:
https://github.com/nathanathan/ContextCommand/blob/master/README.md

[sizzlemctwizzle]

users can type in natural language queries

Personally I'm not a fan of natural languages. I'd rather we communicate in C.

so there wouldn't necessarily be any complicated NLP stuff involved.

As long as it didn't involve NLP and has potential to improve the user experience, I'd be interested. However, the "triggers user script" part sounds like it goes beyond user script repo and into the browser extension territory. The extension could obviously be deeply tied into the site via API.

https://github.com/nathanathan/ContextCommand/blob/master/README.md

Looks interesting. I'll have to give that a read.