I'd like to thank my mentors Connor Burns and Fons van der Plas, the organizations Pluto and Julia, and the program GSoC run by Google for providing me with this wonderful opportunity.

Pre-requisite information

I guess we should first talk about a few terms I might use throughout this blog:

• Julia: Julia is a fast, dynamic, reproducible, composable, general and open-source programming language, as described by its website. It's major use is in mathematical computing, data visualization among others.
• Pluto: Pluto Notebooks are lightweight reactive notebooks for Julia. If you are coming from the python ecosystem, you'll find Pluto and Jupyter similar but different in their own ways. Pluto notebooks offer dynamic environments, interactivity, built-in package manager and HTML interactions among other features.

My Project

So I am trying to build a desktop client for Pluto. My GSoC project description (as quoted below) might explain the problem and the solution in the best way possible.

Problem

Installing Julia, opening a terminal, running Julia REPL, then opening the package manager, adding Pluto package, running Pluto, and then going to the browser to run it. How many steps was it? 7. Seems quite uninviting. Now think of a person who has never touched any terminal in his life. Seems like an even more daunting task. A person who is only interested in running a Pluto Notebook and running Julia on it may or may not know how to do any of these steps, and since these can be automated, why keep this task to the end user?

My Plan to solve

This is where this app comes in, bundles all these steps, runs this in the background, and then only shows the user their notebook running inside the app itself, no browser required.

Now as you might've guessed, my project does not deal directly with Julia. And it might seem it is just a desktop wrapper around Pluto. To some extent you are right, but it is not entirely the case. See my task is to basically wrap all the steps required to run Pluto, and do it on one click, and also to allow Pluto to support desktop APIs via Electron. This will abstract most of the manual steps and will help the user directly get up and running.

Progress so far

Throughout the process, me and my mentor(s) had weekly meets to discuss about the process, future updates and any issues that I may have encountered in development during the week.

Community bonding

• Participated in general discussions in the community meet.
• Discussed with my mentors about various features that they needed and some that I thought about after submitting my proposal.
• Also got a better idea about the scope of the project, the exact target audience and how this aims to help them.
• Did general research about how to implement certain features, and set up the right development environment.

Coding Period

My contributions are in 2 repositories: PlutoDesktop and Pluto.jl.

All these seven letter codes like 12f2836 are commit codes, click on them to get exact code changes.

Week 1

Although the coding period was supposed to start from July 13, I started a bit early (first commit was on June 8) to make myself familiar with the environment and select the right template to go ahead with.

PlutoDesktop

• Starting off with 12f2836, I chose a combination of electron, react and vite using the electron-vite-react template but ran into some problems, like page not being rendered in production build. I could've just tried to find out a way to make it work but then I found Electron React Boilerplate which is much more feature rich, hence I switched to it in aed6480. The same commit also included a script to run Pluto from behind the scenes and extract the URL and load it in an electron window.
• Major functionality was coded in 163926e and 200da4c where I coded the basic Pluto abstraction functions that could open, create and export Pluto notebooks (which we will call FileSystem functions from now on), and added a bit of CLI support.
• 4a0c4ca bundled a Julia binary along with the executable. The application could now extract the Julia zip file into the resources folder and run Julia binary from there, taking in the direction of being an all-included solution.
• c4854cd started exposing the initial FileSystem functions to render process.
• 43ae7ba added multi-window support, the user could now run more than one application instances simultaneously.
• A loading screen was added in e7fec71 which is visible until the Pluto process is up and running in the background and we've captured the required URL to load.
• One initially unnoticed mistake was that even if Pluto application was killed, for some reason the internal Pluto process kept running in the background in some cases. 02ca306 fixed this by force killing the Pluto process before the application is exited.

Pluto.jl

• Created a draft pull request desktop-support in order to help the frontend make use of API provided by the PlutoDesktop environment.
• 034b048 overwrote some of the functions that are implemented by directly calling the Julia backend to rather go through the API provided by the application. It also did some UI changes for the desktop version of the frontend.

Now we had something that in spite of having a lot of bugs and errors, could serve a very basic user quite smoothly.

Week 2

PlutoDesktop

• 1558fb2 added 2 more FileSystem functions: "move" to move a Pluto Notebook and "shutdown" to shutdown a Pluto Notebook. Moreover every time the window having a notebook opened inside it is closed, that notebook is shut down automatically.
• Now one problem I came across was that if you install a production build, it installs in C:/Program Files/Pluto, turns out you cannot "write" in that directory without admin powers. dee8a7b fixed this by asking for admin powers only when the Julia zip was needed to be extracted, which generally happened only on first run.
• dff2363 gave the application the ability to open Pluto notebooks having all the extensions supported by Pluto.
• From bb98dd0, the application started showing a loader for every new window opened.
• 9f25ca6 added support for PDF export.
• 67cccec renamed the API available to render process from electron to plutoDesktop because electron is a very common API name, hence could be mis-interpreted for some other platform.

Pluto.jl

• 7c61ae2 allowed the frontend to use the new "move" function provided by the API.
• Now one issue was that when you move a file, an event called "update-notebook" is sent to the Julia backend from the frontend. But this was already been done by the FileSystem! Hence this was causing inconsistencies between the frontend and the backend. Fixed this in f7b473a.
• The Pluto.jl repo has type declarations inside the JSDoc comments, this works fine for function parameters but there was no way to declare a global type for window.electron so that TypeScript rules could calm down and not leave me with a single option of using // @ts-ignore to get out of all the hassle. dafec3a (duplicate of 929e6ad) tackled this by creating a types.d.ts file, globally declaring window.electron type and allowing for global type declarations in the future as well.
• 3a5a649 refactored FilePicker component to use an extra on_desktop_submit prop.
• a8647c3 changed the API name from electron to plutoDesktop in the frontend as well.

Week 3

PlutoDesktop

• With 5d01b92 I finally had all the Pluto icons in place.
• Added "Bug Report" and "Feature Request" issue templates in 213913d. Such a nice feature from GitHub!

Pluto.jl

• Added analytics support for desktop applications! From c647dda, desktop analytics can be seen as \desktop\* on Pluto Stats GoatCounter.

A few bug fixes here and there, and the first release was ready: v0.0.1-alpha pre-release.

Rest of the week was spent on gathering reviews and feedback on the release from the Pluto.jl Development Channel and a few others and discussed with the mentor about possible solutions.

Week 4

PlutoDesktop

• This week was spent fixing a few bugs discovered from the received feedback, and refactoring the code a bit.
• The only new feature was added in 43925bc which allowed the Pluto Frontend to export a notebook via methods provided by the plutoDesktop API instead of doing it normally, because it was resulting in an error.

Pluto.jl

• Added a new hook usePlutoDesktop to handle plutoDesktop function calls in f6e5441.
• c98da06 allowed the frontend to export using the FileSystem functions.

Week 5

• One of the major issues reported in the feedback by many people was the need to give admin permissions to extract the Julia zip. Now that was not really a bug, but it just broke the process a little, and it was not as smooth as it was supposed to be. Majority of this week was spent trying to find a workaround.
• Major breakthrough was achieved in 521443a where the application was now bundling the extracted folder instead of the zip. I assumed that it would result in an increase in the installer size but to my surprise it was exact opposite! The new installer was around 33 MB smaller than the previous iteration and it removed the need for admin permissions as nothing was being extracted, win-win!
• One more requested feature was the ability to use a custom Julia binary instead of the one bundled with the application, this was also fixed in 521443a only. Now one could give the correct value to the key CUSTOM-JULIA-PATH in the config file, and the application would now give priority to this path over the default JULIA-PATH.
• A step was introduced in the bundling process done by electron-builder in 4d6ba6e which would extract the zip and overwrite the existing the extracted folder to bundle with the application automatically.
• A draft release was created in order to test this new feature out in different systems.
• 1c63734 added the support to open notebooks via given URL instead of local file path.

Week 6

PlutoDesktop

• A bit of refactoring was done here (bc6a446) and there (5f78f65).
• MAJOR PULL REQUEST WAS MERGED! and yes I am talking about the new installation process. I checked its working thoroughly and got approval from my mentor regarding it.
• Minor improvements were made in 59a0032.
• Another new feature was implemented in f49bcac. This one added pre-compilation. What happens is that when you normally cold-start Pluto from Julia, it takes a few seconds to half a minute to get ready and running. Although this time might be acceptable to some people, a little speed never hurts ;) Here is where the pre-compilation step comes in. basically Julia allows you to save loaded packages and compiled functions into a system-image using a package called PackageCompiler. You can then use this system-image to cold-start Pluto and it is fast, much faster than before. And the pre-compilation is done on a separate thread in the background, thus, there is no extra load on the main thread!
• d673d5c is one of my favorite commits. I never thought of logging to be like, one of the most crucial parts of developing a software. Like alright you need console.log to see what is happening somewhere and that is pretty much it, right? Things changed when I saw how huge my logs were becoming, moreover I couldn't realize which log was coming from where, whether it was from Electron or Julia. This commit added 3 different loggers, which created beautiful readable logs in 2 separate log files. Neat, isn't it? Have a look:

Week 7

PlutoDesktop

• 4659c67 assembled all the separate functions related to Pluto and Julia into one single class. Nice OOP vibes.
• Minor bugs and fixes here and there.
• ddf2edd added logs for axios in dev mode.
• dc88668 removed the extra loading window and added a loading screen in the main window itself.

Week 8

PlutoDesktop

• e456523 added a host of fixes and added support for opening a file from URL.
• 579b1ee was the highlight of this week for some reason.

  await (await import('./devtools')).default();
  

  has to be the cleanest line of Js I have ever written. It imports the function that launches the devtools and executes it, in one line.
• From 7062410, the application asks the user for admin rights, if precompilation is needed.

Week 9

PlutoDesktop

• Some minor fixes.

Week 10

PlutoDesktop

• 4ec5c85 added some much needed documentation for future reference.

Week 11

PlutoDesktop

• A new release 🎉🎉🎉, version 0.0.2 was released with a host of improvements over the last version.
  https://github.com/JuliaPluto/PlutoDesktop/releases/tag/v0.0.2

More updates soon...

This blog was last updated on 31-07-2022.