Adding logging

[mcminis1]

#390

[jerryjliu]

I think I'm ok with this! could you provide an example of a user being able to easily use the logger for stdout? I want to make sure that it's still easy to use in a Jupyter notebook setting.

also, are you planning to land multiple PR's?

[mcminis1]

@jerryjliu I updated a notebook, examples/vector_indices/SimpleIndexDemo.ipynb, with an example. Let me know what you think.

I also added a line in the docs quickstart, and updated the rest of the notebooks.

[jerryjliu]

thanks @mcminis1!

just a heads up that i do want to spend some time going through this PR to make sure that the user flows all make sense. i'll have some time to get to this later this week. apologies for the delay in advance, just wanted to let you know!

[mcminis1]

@jerryjliu sure. I like logging for production use cases. For the notebook, I agree, it feels a little weird. I think they two of us have slightly different perspectives on library design. I'm probably more concerned about use in an application and you're more interested in use in a notebook.

Usually objects have string methods def __repr__(self) -> str: so that, in a notebook, if the object is the last thing in the cell it returns a string. The ntermediate progress isn't output, but the final thing is. If you want something as an intermediate output, then you would structure the object in such as way that you could get it. For instance, consider how requests has a prepare method. Using that you can see all of the intermediate parts before executing the call.

So, after you think on it let me know what you want to do. Perhaps a good middle ground is leaving all the verbose stuff in the code and just adding in the logging as it is here. then the notebook experience is identical, but you can still get good logs.

[jerryjliu]

@jerryjliu sure. I like logging for production use cases. For the notebook, I agree, it feels a little weird. I think they two of us have slightly different perspectives on library design. I'm probably more concerned about use in an application and you're more interested in use in a notebook.

Usually objects have string methods def __repr__(self) -> str: so that, in a notebook, if the object is the last thing in the cell it returns a string. The ntermediate progress isn't output, but the final thing is. If you want something as an intermediate output, then you would structure the object in such as way that you could get it. For instance, consider how requests has a prepare method. Using that you can see all of the intermediate parts before executing the call.

So, after you think on it let me know what you want to do. Perhaps a good middle ground is leaving all the verbose stuff in the code and just adding in the logging as it is here. then the notebook experience is identical, but you can still get good logs.

Yeah I don't disagree. I do want to make this good for application use - mostly just trying to think about how to also keep the notebook experience somewhat seamless. I will have some thoughts today or tomorrow!

[jerryjliu]

I just played around with it - actually I like this! I'm going to take a deeper look tonight. Thanks for putting together this PR @mcminis1, this is going to be super valuable

[jerryjliu]

heads up: i'm pushing some changes to fix linting. afterwards i'm down to land!

one followup after this, is that some logging uses the root logger whereas others uses the module logger. we should refactor to be consistent (and use module logger everywhere probably).

[jerryjliu]

congrats!

what's your twitter? happy to give you a shoutout on the release tomorrow

[mcminis1]

https://twitter.com/JeremyMcminis

thanks!