Make GI scripts working again

[tknopp]

The introspection scripts do not work anymore since
https://github.com/bfredl/GI.jl
is not Julia 0.5 ready.

I made a fork and did try to fix things in a try and error manner:
https://github.com/tknopp/GI.jl

Still not fully their. @bfredl: Interested in making these working again?

[tknopp]

I have been working getting GI.jl running under Julia 1.0 but I still face an issue

ERROR: LoadError: UndefVarError: GLib not defined

when I execute GI.jl/test/test_gtk.jl

[bfredl]

@tknopp Did the structure of Gtk.jl change? I think it wants to import Gtk.GLib as GLib.

[tknopp]

Glad to hear from you. Have a look here:

https://github.com/tknopp/GI.jl/blob/master/src/giimport.jl#L24

I actually tried making it a "using Gtk.GLib".

[tknopp]

and peval gives me

$(Expr(:toplevel, :(module _Gtk
  #= /home/knopp/.julia/dev/GI/src/giimport.jl:8 =#
  #= /home/knopp/.julia/dev/GI/src/giimport.jl:8 =#
  using GI, _AllTypes
  using Gtk
  using Gtk.GLib

[bfredl]

Yeah sorry for my inactivity, I'm being stuck with python/tensorflow codebase for my current research projects. But I hope to get back into julia-land soon and consider using Flux.jl for my next project 😅

The code you pointed to is manually constructing the AST, probbaly the julia AST has changed quite a bit. Especially Expr(:using, Symbol("Gtk.GLib")) look suspicious, Gtk.GLib is probably not a single symbol anymore. maybe it can be replaced by :(using Gtk.GLib), unless hygiene gets in the way.

[tknopp]

The Symbol("Gtk.GLib") was me. Before peval reported using Gtk,GLib which is certainly wrong. You can see my changes here:

bfredl/GI.jl@master...tknopp:master

I think there were changes to eval in 1.0 but I don't feel a home in that area.

In general it would be great if we could get GI.jl running again since this seems to be the best approach to move Gtk.jl forward.

[bfredl]

Still, I'm pretty sure Symbol("Gtk.GLib") is wrong. :(using Gtk.Glib)).args gives Expr(:., :Gtk, :Glib)

[tknopp]

if I do

decs = Expr[ Expr(:using, :., :., :GI, :_AllTypes), Expr(:using, :Gtk, :. , :GLib) ]

I get

$(Expr(:toplevel, :(module _Gtk
  #= /home/knopp/.julia/dev/GI/src/giimport.jl:8 =#
  #= /home/knopp/.julia/dev/GI/src/giimport.jl:8 =#
  using ., ., GI, _AllTypes
  using Gtk, ., GLib
...

[bfredl]

But it should be Expr(:using, Expr(:., :Gtk, :GLib)) based on what I just said.

[tknopp]

ok that seems to bring us further. Now I get

ERROR: LoadError: invalid subtyping in definition of GtkWidgetAccessible

[bfredl]

Not sure, I think I'm relying on Gtk.GLib to actually generate the wrapper types. But maybe GI.jl is feeding it bad data.

[tknopp]

yes it seem so. It might be an ordering issue. I think the concrete type is defined before the abstract one.

[jwahlstrand]

Here is an experimental, very rough collection of mostly automatically exported GObject-based packages:
GLib.jl - includes libgio
Pango.jl - includes libpangocairo
GdkPixbufLib.jl
Gtk3.jl - includes libatk, libgdk3
Gtk4.jl - includes graphene, libgsk, libgdk4

These are generated by GI.jl. GLib.jl is basically the GLib/ subdirectory in Gtk.jl with a few changes and its own GI-generated structs and methods.

GI.jl now does what I personally wanted (help me use a particular GObject-based C library) but many aspects are untested. Many methods are broken, especially anything related to signals and callbacks. Handling of structs with fields needs more work.

So, what's the best way to start incorporating introspection into Gtk.jl? Constants and enums are already automatically generated and GI.jl doesn't improve on that. I think the lowest hanging fruit is automatically outputting all objects, interfaces, and boxed types (replacing the macros in gtype.jl). But that arguably isn't much of an improvement over the current approach. It would help with wrapping new libraries though. GI.jl also outputs object property info, for example the default type of object properties, and defines getproperty() and setproperty!() for GObjects, so you can for example set a property using my_object.property = "foo". Once that is more tested it could be useful.

Outputting methods is probably where GI could help the most, with more work. The generation scripts currently try to export everything (skipping quite a few to avoid method collisions and problematic argument types), but for real packages it would probably be better to export curated lists of methods that are known to work. Julia methods like push!(), etc. could call these automatically generated methods instead of using ccall(). The benefit GI would provide would be using annotations that specify whether the caller needs to free return values (preventing memory leaks), whether arguments or return values can be NULL, etc.

I'm a Julia beginner and am sure that massive improvements could be made by someone who knows what they're doing. I'll keep tinkering, as time permits, but if there is interest in applying this to Gtk.jl, I'd be happy to help and/or pass the baton.

[tknopp]

Does this mean that you got also Gtk4 working? That would be really awesome.

My thoughts are the following:

1. In my opinion we should have two layers. The first are just the raw 1to1 wrapped methods, without any modification. These should be automatically be generated as you do. The second layer would be translation area where we would take the raw methods und transfer this to the public interface. This would certainly also done white lists and code generation. So many methods can be directly but some also need some translation (1 based indexing is one example).
2. When we go done this route we would get rid of the GAccessor module which is IMHO the right direction. We should just have one public API which does not expose to the user, whether a function has been manually or automatically been wrapped.
3. It is not clear to me how your approach fits to the GLib module where the mapping of Gtk to Julia types is done. This is right now that part of our code base that is mostly magic to me.

It would be very much appreciated if you could work on this further since Gtk.jl has not really a maintainer right now. @timholy @vtjnash @jonathanBieler: Whats your take on this?

• Should all the packages be splitted? Or should probably there should be one repository with multiple packages in it?
• Try to be API compatible or start with fresh approach?
• Gtk3/4 in individual packages or trying to put both together?

@jwahlstrand: I invite you to JuliaGtk, where we can later move all these packages when we have a plan.

[jwahlstrand]

Pieces of Gtk4 work -- 99.9% is untested. GI.jl's method output works for quite a few types of arguments but I keep encountering methods that are broken.

On your point 3, right now GI.jl doesn't use the macros in gtype.jl but outputs similar code for each GObject, GInterface, etc. in a namespace all in one go. I made some changes to support the "transfer ownership" annotations in GI and these should be checked carefully. I'll summarize the differences when I have more time.

These could definitely be put into one package rather than split like this, but Gtk 3 and 4 are fairly different so I think they would almost have to be in separate submodules.

[tknopp]

how do you develop these packages? I wanted to try it out but it was not possible to dev them through the package manager.

[jwahlstrand]

They're not registered. Should I do that?

[jwahlstrand]

I see my mistake, I'll fix the project files...

[tknopp]

No registration is not necessary at this point. But having proper Project.toml files allows for make a local environment in which the packages can then be tested.

[jwahlstrand]

They should hopefully work now... Note that Gtk 4 relies on the library being present outside Julia. I've only used it on Fedora and the library might have a different name on other platforms.

[tknopp]

I just tried this out with GTK3 and it seems to work quite well. So I would definitely support that you push this forward.