Skip to content

Meeting Notes 2025 #73

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
Zsailer opened this issue Jan 2, 2025 · 13 comments
Open

Meeting Notes 2025 #73

Zsailer opened this issue Jan 2, 2025 · 13 comments

Comments

@Zsailer
Copy link
Member

Zsailer commented Jan 2, 2025 •
edited
Loading

Hello everyone,

Gitter

Welcome to the Jupyter Server Team!

We meet on Thursdays at 8:00am, Pacific Standard Time on Jupyter's Zoom Channel.You can add yourself to the weekly agenda here. Everyone is welcome!

Let's avoid using this thread for discussion. If you'd like to discuss something in the minutes, open a separate issue and reference this thread.

You can find previous year's notes here: 2020, 2021, 2022, 2023, 2024

Meeting Notes
@Zsailer Zsailer pinned this issue Jan 2, 2025
@Zsailer
Copy link
Member Author

Zsailer commented Jan 16, 2025

January 9th, 2025

Name affiliation GitHub username
Vidar T Fauske JP Morgan Chase @vidartf
Ian Thomas QuantStack @ianthomas23

Agenda

@Zsailer
Copy link
Member Author

Zsailer commented Jan 23, 2025

January 16th, 2025

Name affiliation GitHub username
Zach Sailer Apple @Zsailer
Andrii Ieroshenko AWS @andrii-i
Luciano Resende Apple @lresende
Vidar Tonaas Fauske JP Morgan Chase @vidartf
Piyush Jain AWS @3coins
Ian Thomas QuantStack @ianthomas23
Omar Jarjur Google @ojarjur
Afshin T. Darian QuantStack @afshin
Sofus Rose Just Me @so-rose
Jan-Hendrik Müller Also Just Me @kolibril13

Agenda

  • SSC Representative Nominations.

    • Email sent this morning
    • Nominations are due Jan 29th, 2025.

  • Andrii

    • [jupyter-collaboration] Issue #390 "notebook and lab can't open in RTC" - I can reliably and easily reproduce this problem by updating from jupyterlab-chat version that uses jupyter-collaboration v2 to one that uses jupyter-collaboration v3; .jupyter_ystore.db file is not getting moved / updated notebook and lab can't open in RTC jupyterlab/jupyter-collaboration#390 (comment)

      • (discussion from JupyterLab call)
        • David Q: We should explore dropping YStore by default.
        • We should open new issue for a) adding a configurable for enabling YStore & defaulting to False, and b) making YStore better. Instead of a document_ttl, we should probably just have max_updates. This bounds the disk use and makes disk use independent of when the updates were made.
      • [Zach here] my hunch is that this thin package might be a culprit. I'm not saying this package is a problem, but it's heavily under-tested. lihttps://github.com/davidbrochart/sqlite-anyio

    • [jupyter-collaboration] Issue #425 "Should most collaboration python packages depend on jupyterlab?" Are there any downsides to adding a JupyterLab dependency pin to jupyter-collaboration? A mechanism like this to specify which versions of JupyterLab current version of jupyter-collaboration is compatible with would help with versions mismatch issues. Should most collaboration python packages depend on jupyterlab? jupyterlab/jupyter-collaboration#425

      • From disucssion during JupyterLab call community seems okay with pinning, looking for more opinions if any, would be happy to open a PR

  • Piyush

    • Is there a plan to move discussions to Zulip from Gitter?

  • Jan-Hendrik, Sofus: Demo Blender+Jupyter: https://github.com/Octoframes/bpy_jupyter

    • Such a cool demo!
    • Flags for enabling Blender to connect an external kernel 
      --ServerApp.allow_external_kernels=<Bool>
    Whether or not to allow external kernels, whose connection files are placed
    in external_connection_dir.
--ServerApp.external_connection_dir=<Unicode>
    The directory to look at for external kernel connection files, if
    allow_external_kernels is True. Defaults to Jupyter
    runtime_dir/external_kernels. Make sure that this directory is not filled

@Zsailer
Copy link
Member Author

Zsailer commented Jan 31, 2025

January 30th, 2025

Name affiliation GitHub username
Vidar T Fauske JP Morgan Chase @vidartf
Omar Jarjur Google @ojarjur
Ian Thomas QuantStack @ianthomas23
Mike Krassowski
Zach Sailer

Agenda

@Zsailer
Copy link
Member Author

Zsailer commented Feb 13, 2025

February 6th, 2025

Name affiliation GitHub username
Vidar Tonaas Fauske JP Morgan Chase @vidartf
Afshin T. Darian QuantStack @afshin
Luciano Resende Apple @lresende
Ian Thomas QuantStack @ianthomas23

Agenda

@Zsailer
Copy link
Member Author

Zsailer commented Feb 20, 2025

February 13th, 2025

Name affiliation GitHub username
Vidar T Fauske JP Morgan Chase @vidartf
Omar Jarjur Google @ojarjur
Luciano Resende Apple @lresende
Ian Thomas QuantStack @ianthomas23
Harsh Parekh None @hXtreme

Agenda

@Zsailer
Copy link
Member Author

Zsailer commented Apr 3, 2025

February 20th, 2025

Name affiliation GitHub username
Zach Sailer Apple Zsailer
Ian Thomas QuantStack @ianthomas23
Harsh Parekh None @hXtreme
Vidar T Fauske JP Morgan Chase @vidartf
Gordon Bean Brigham Young University CS @brazilbean, @byubean
A. T. Darian QuantStack @afshin
Jayan Kuttagupthan AWS @kjayan

Agenda

  • Ian:

    • Support for particular (old) versions of python?
    • Stop support for Python 3.9 ipython/ipykernel#1316
    • Decision for Jupyter as a whole? Or could just be kernel decision.
    • e.g. IPython drops early, Spyder prefers not to.
    • Believe there was a previous discussion about following NEP29, maybe in some governance repo.

  • Should we cut a release of jupyter-server?

    • Main PR driving this release is

@Zsailer
Copy link
Member Author

Zsailer commented Apr 3, 2025

February 27th, 2025

Name affiliation GitHub username
Vidar T Fauske JP Morgan Chase @vidartf
Omar Jarjur Google @ojarjur
Patrick Chen Google @ptch314
Luciano Resende Apple @lresende
Harsh Parekh None @hXtreme

Agenda

  • Patrick

    • % in error message being encoded into %% (by Tornado HTTPError) jupyter_server#1503
    • is this a bug or intended behavior?
      • The web.HTTPError constructor is documented to support %-style format strings with additional args used for formatting.
      • Calling that constructor without additional args (such as when calling with a non %-style format string, like an f-string) results in it escaping all % characters with %%.
      • This can result in the message passed to the constructor getting modified/corrupted.
      • Throughout the Jupyter server codebase the web.HTTPError constructor is called with a mixture of f-strings, %-style format strings with args, or %-style format strings that are formatted before passing.
        • Any of the calls without args (f-strings or already formatted %-style strings) might be affected by this same doubled-% issue.
        • Even the Tornado codebase seems to be inconsistent about this.
    • Consider engaging with Tornado about the HTTPError behavior

  • Jayan:

  • Harsh:

    • Will create a PR on Tornado for the HTTPError log_message args formatting

@Zsailer
Copy link
Member Author

Zsailer commented Apr 3, 2025

March 6th, 2025

Name affiliation GitHub username
Vidar T Fauske JP Morgan Chase @vidartf
Jayan Kuttagupthan AWS @kjayan
Harsh Parekh None @hXtreme
Ian Thomas QuantStack @ianthomas23
Drake Aiman Google @wh1ter4bb1t-js

Agenda

@Zsailer
Copy link
Member Author

Zsailer commented Apr 3, 2025

March 13th, 2025

Name affiliation GitHub username
Vidar T Fauske JP Morgan Chase @vidartf
Zach Sailer Apple @Zsailer
Omar Jarjur Google @ojarjur
Ian Thomas QuantStack @ianthomas23

Agenda

  • Nick
    • Send2Trash discussion
    • If configured to send2trash, you won't know if it will work until you try. The API doesn't give you enough information to know if it will work.
    • What's the UX we should do if it fails?
      • Say Send2Trash failed, do you want to hard-delete?
  • Harsh (can't attend zoom call)
    • Progress report on: 1503 -- need to adress a backwards compatibility concern for tornado other than that we should be good

    • What is jupyter doing with these exceptions that's showing unprocessed HTTPError.log_message to users?

    • If someone can help respond to the above question
      • Omar: I'll respond to that question. TL;DR - that output gets written to an HTTP response body and then used by JupyterLab to generate an alert dialog.
      • Vidar: I would put in a regression test for the issue as well. That would help the review understand what is wrong with with current issue. E.g. what happens for an excpetion with a URL with % in it?
  • Omar
  • Welcome, Ian Thomas, to the Jupyter Kernels Team: Add Ian Thomas to the Jupyter Kernels Team! jupyter/kernels-team-compass#15

@Zsailer
Copy link
Member Author

Zsailer commented Apr 3, 2025

March 20th, 2025

Name affiliation GitHub username
Vidar T Fauske JP Morgan Chase @vidartf
Mike Krassowski Quansight @krassowski
Omar Jarjur Google @ojarjur

Agenda

@Zsailer
Copy link
Member Author

Zsailer commented Apr 3, 2025

March 27th, 2025

Name affiliation GitHub username
Zach Sailer Apple @Zsailer
Jason Grout Databricks @jasongrout
Omar Jarjur Google @ojarjur
Ian Thomas QuantStack @ianthomas23

Agenda

  • Jupyter Server Terminals:

  • What have been folks experience on AnyIO? Zach is reading through the code and realizing the code is quite complicated. Zach says that exceptions grind everything to a halt, for example, in RTC,

    • History of anyio, according to Nick Bollweg: this solved issues on Windows around async file i/o with the standard lib. For example, see here and here
    • From Ian: AnyIO is actively maintained (and thus changing...). People that spend lots of time in Python async become trio fans, but the rest of us using native async/await are left out.
      • ipykernel 7.0 is still not being released (going on 6 months delayed), and part of the problem is anyio exception handling and anyio changes.
      • Ian is biased against anyio, since he is stuck on his subshell contribution partly based on being blocked on anyio being good enough.
      • There is a tendency when you are proficient in AnyIO to create lots of tiny threads to handle events. Zach concurs.
      • If Ian was designing a system from scratch, he would look at AnyIO and try to design in the system. However, converting async/await to AnyIO seems quite difficult.
    • Nick: a few testing tools might help, like Blockbuster and BearType
    • pycrdt depends on anyio, independent of jupyter server.
    • example problem in ipykernel
    • Nick suggests potentially using hypothesis or locust. Nick sets up testing rigs like this in his day job. This can stress-test the system and potentially expose problems that happen at load.
    • Also introducing comprehensive logging into the RTC system (potentially that you can switch on when needed) to debug state leading up to problems.
    • Who is working on pycrdt? David Brochart, to some extent Mike, and Zach's team contributed exception handling code. Pycrdt depends on y.rs, and IIRC Kevin still maintains y.rs. The data corruption problems are (likely?) on the Jupyter side.

  • Meeting questions: Should we combine this meeting with the frontends call on Wednesdays?

    • There are lots of meetings, and the groups are smaller than they used to be. Perhaps it is better to consolidate to have more overlap in conversations? We might schedule the calls back-to-back to make it easier to block off time and have overlap.
    • +1 from Ian, Jason

  • (if time) socializing openssf baseline, FAQ

  • Saved for next time: a discussion on dependency injection for Jupyter server plugins - we are seeing Jupyter plugins that depend on other plugins, but don't have a nice way for that dependency happening.

@Zsailer
Copy link
Member Author

Zsailer commented Apr 3, 2025 •
edited by afshin
Loading

April 3, 2025

Name affiliation GitHub username
Zach Sailer Apple @Zsailer
Vidar T Fauske JP Morgan Chase @vidartf
Drake Aiman Google @wh1ter4bb1t-js
Ian Thomas QuantStack @ianthomas23
Omar Jarjur Google @ojarjur
Afshin T. Darian QuantStack @afshin

Agenda

  • PyCRDT to Yjs org.

    • Move pycrdt to y-crdt organization #77
    • David opened the discussion. Those on the call agreed it made sense.
    • Darian: Let's start a vote among the team AND also ask if @dmonad and @Horusiath to vote on it. (This is unprecedented, but would be a good thing to record since we're handing off the project.)
    • Questions:
      • We just moved PyCRDT to the Jupyter PyPI org, which means there's a list of people who have publishing rights today. Do we keep it here? If not, I (Zach) don't think it's necessary to keep all all of Jupyter org maintainers as maintainers on the PyPI project. Shoudl we list a subset of people to be added there?

  • Ian

    • Further thoughts on anyio and subshells in ipykernel following on from last week's discussion.
    • anyio was the answer to replacing tornado with something else that is actively maintained and supports trio.
    • Ian proposes to try subshells in ipykernel on 6.x branch, so avoiding anyio, initially just to see if it is possible.
      • Zach stated support.
      • Currently, the anyio work is in the main branch. We might need to move off that work to a different branch and free up main again.
      • Once this work is in a working state, will need to coordinate with someone to help with the branch management.
    • Ian will open an ipykernel issue to communicate my intention to other interested parties.

  • Discussion around evolving old codebases (sparked by Ian's ipykernel topic)

    • ipykernel is an example of a project that is difficult to evolve because it's supporting a lot of use-case; however, it isn't clear that all of these use-cases are still relevant.
      • For example, the anyio work is happening so that we can support trio, but do we have clear signal that many people are looking for trio support in ipykernel? If not, is the anyio work required? It's a lot of code that we would need to commit to maintaining.
    • jupyter-server's kernel API likely needs a revamp; however, backwards compatibility has always been a blocker. It's not clear that we need to support old patterns anymore. How do move forward from here?
    • Vidar: Generally, a good pattern is to use two major releases to cover a major change. The first one allows folks to use both the old and new in the same release. That way, they can iteratively migrate.
    • JEPs are a good place to make proposals about dropping support for things. Even if they don't get accepted, it triggers discussion.

@ianthomas23 ianthomas23 mentioned this issue Apr 4, 2025
Open
@Zsailer
Copy link
Member Author

Zsailer commented Apr 10, 2025

April 10, 2025

Name affiliation GitHub username
Zach Sailer Apple @Zsailer
Vidar T Fauske JP Morgan Chase @vidartf
Drake Aiman Google @wh1ter4bb1t-js
Ian Thomas QuantStack @ianthomas23
Jason Grout Databricks @jasongrout

Agenda

  • We had a conversation about the challenges of volunteer work in open-source.

    • Stages of contributing to open-source from Ian:
      1. Fun! Contributing is fun and exciting, learning and developing new things and contributing to the world
      2. Responsibility! You're now maintaining stuff you have built, keeping it from breaking, building out its impact. This is the really hard stage where we see burnout
      3. Elder Statesperson - you're in the room by reputation, people come to you with questions and respect your judgement and opinion, but often others are doing the actual work.

  • Make Jupyter Server terminals authorization async: allow async authorization jupyter_server_terminals#109

    • Making a release today.

  • Move pycrdt to y-crdt org. Vote open: Move pycrdt to y-crdt organization #77

  • New release of jupyter-collaboration + ipywidgets happened yesterday.

  • Discussion around a framework for jupyter server extensions providing services for other extensions. Conversation centered around OpenAPI and AsyncAPI as specs for the message protocol. Zach is working with a student on a project around this.

  • We should use -Werrror or --pythonwarnings=error on our tests: Turn on -Werror in our unit tests jupyter_server#1512

    • In the past, deprecation warnings made this unworkable, making it hard to merge PRs.
    • Perhaps we lock down the dependencies for a testing environment, and we can ignore specific deprecation warnings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@Zsailer