Proposition: to separate addons from Io

[ales-tsurko]

Hi Steve! As I can see the main problem with building Io are its dependencies. When some of addons won't build, a user didn't get Io too. Also not all the addons equally important for all users. It would be better to allow user to install what he/she needs. So why don't separate the addons from Io? We could convert the addons into Eerie packages. And make a repository for the each one. A user then could install what (s)he needs with Eerie. This also would make the Io's build process a lot easier, reduce dependencies and make Io itself smaller. And I'm not speaking about such features as addons updates and environments.
Moreover, Eerie not only makes it simple to install Io addons and all of its dependencies, but it's also has a Docio — the documentation generator for the packages. Docio generates documentation from the same comments format, that is already used in addons, but also supports markdown. After the generating process it provides you with a site for a package (this site is generated by Docio, for example: http://tonikasoft.com/docio/) and methods to look up for an addon documentation right in the Io's interpreter.
I could convert all the addons into packages right after we'll finish fixing cross platform problems (there are not many).
This also assumes that Eerie will be the official Io's package manager, that will be installed with Io itself.
What do you think?

[stevedekorte]

Hey Ales, I'm open to patches that would make Io easier to install. I'm also open to alternative ways to generate docs as long as they are in the same format as the current docs.

[ales-tsurko]

Great! Could you then make repositories for the each of the addons? Then I could fork them and make PRs to make them packages. Or you could make an organization for the addons (and, maybe, Io itself) and temporally allow me rights to create repositories in there.
Otherwise I should make it under https://github.com/tonikasoft profile.

[ales-tsurko]

The repositories may not contain anything — I'd copy the sources myself. But they should be properly named (had the same names as in the addons directory).

[stevedekorte]

Why are separate repos required?

[ales-tsurko]

Eerie keeps the package URL so you can update the package. It's also would be simpler to install packages separately.
For example, I'd like to install AVCodec addon, I'd do it so:
eerie install https://github.com/AVCodec.git
Then if I'd like to update it, I'd just call:
eerie update AVCodec
And Eerie would download the updated code and will recompile it if needed.
Not only Git supported, but also svn, mercurial, archives and local directories. But there is a drawback with the last one (local packages). They are good for development or, for example, to convert some Io project into a package. But if we ship the Io's source code with all the addons (so they all be installed from local directories), then in order to update some addon, the user should pull all the changes from the main repository (for all the addons and Io itself) and only after that (s)he could call eerie update for a specific addon. This is because eerie will update the addon using installed URL, that will be a local directory in this case.

A separate repository is just its own place. It makes it independent from the other addons and Io's sources.
Don't worry — if a package depends on another package, eerie will fetch and install it automatically from its repository. And there will not anymore a situtation when to compile Io, a user would have to install dependencies for all of the addons.

There are also other benefits, for example all the issues regarding specifical packages goes to each own repository. It would be also simpler for contributors to work on a specific addon on a separate repository.

[ales-tsurko]

That also corresponds to "encapsulate what varies" principle in some kind.

[ales-tsurko]

I mean to allow a user to choose which addons to install. And by default there would be not many addons — just those, that required. After installation a user could install any additional addons (s)he needs (or don't install anything). But if you'd like it's not so hard to add all of them as dependencies, so they all will be installed, in this case Eerie at least would allow a user to remove what (s)he doesn't need.

Eerie packages have the same directory structure as the existing addons, except of the package manifest file (package.json). So it'll be not so hard to convert them. Eerie also uses the good old (and fixed) AddonBuilder to build native packages that makes it independent of and simpler to use than Cmake.

Packages could contain an executable scripts, so they could be installed as apps. For example, jasmineio package is shipping with jasmineio command line tool: https://github.com/bekkopen/jasmineio
In this sense Eerie is more like homebrew for Io — not just an "addon builder and updater". That means that Eerie is also a manager for apps written in Io (i.e. a package could be not just an addon, but a whole application or game).

There are a lot of benefits of this design. Do you see any drawbacks?

[ales-tsurko]

While fixing one of the addons, I've found yet another benefit: there is no need to recompile and reinstall Io with Eerie — just the package:)

[stevedekorte]

Ales, Sorry for the delay responding. This sounds promising.

One concern I have is if the API for addons changes, it would require dealing with a lot of different repos to update everything. I don't see that happening though, so maybe it's not an issue.

Another concern is that it would clutter my git repo namespace, but maybe it's time to move Io to a separate Io github organization. That might also make granting privileges to maintainers easier too. I'll look into it.

[ales-tsurko]

Even if there are chances that API will change in the near future, the repos are just like the separate directories in the addon directory in that case. The only difference is that all they should be cloned to local directory, but this task could be automated and after that the potential work would be the same as if that happened now.

+1 for organization! I could make all the work with addons then, if you allow me rights to create repositories in there.

[stevedekorte]

I've set up an https://github.com/IoLanguage and made you a member. You should have permissions to create repos for the addons. Want to give that a try?

[ales-tsurko]

Hooray!
BTW, github has an option to move an existing repo to an organization. It's somewhere in settings.

[ales-tsurko]

The plan is next:

1. Right now I'll create empty repos for addons so I could fork and change them into Eerie packages;
2. I'll make a fork of Io, that configured to compile without the addons, but with Eerie;
3. I'll convert all the addons to Eerie packages.

After the last step we could merge the Io's fork.

[stevedekorte]

"BTW, github has an option to move an existing repo to an organization. It's somewhere in settings."

Ok, I'll try to move the repo.

[stevedekorte]

Moved.