Nextflow

Nextflow and nf-core: Hot news!

Reading time: 7 minutes

Read this post in Portuguese by clicking here.

I write this post while I’m still in Spain, in the city of Barcelona. For those who haven’t been following me on social media, in July I finished my PhD in France and went straight to Barcelona, to do the onboarding week at Seqera Labs, the company that hired me to be the Developer Advocate of Nextflow and nf-core for the Latin America region 🌎. After just over a week, I headed to the city where I chose to live: Natal, Rio Grande do Norte, Brazil πŸ₯°

Nextflow is both a Domain-Specific Language (DSL) for writing workflows and a software runtime for running and managing the execution of these workflows. nf-core, in turn, is a community that emerged to curate bioinformatics pipelines written in Nextflow and to work on good practices for the development, maintenance, and use of these pipelines. Today, I’m also part of the core team of nf-core, a team that consists of individuals with a greater degree of involvement and responsibility in the community. But if I returned to Natal after the onboarding week, what am I doing in Barcelona now!? That’s what this post will talk about, looking back over the last 30 days that were incredible and the result of a lot of dedication from dozens and dozens of people!

I can’t lie. It’s obvious that I’m counting down the hours to get back home as my flight was canceled and I’ll need to stay in Barcelona for two more days πŸ˜…. At the same time, my feeling of gratitude and joy knows no boundaries! The last 30 days have been amazing! What a community! At the end of September, the nf-core core team met in Stockholm, Sweden to discuss various topics, touching on important issues from project management itself to technical topics such as sub-workflow management and support for other DSL-version-2 features. (Domain-Specific Language) from Nextflow, as plugins.

Of course, it was also a moment of great integration and fun. What fantastic people! Meeting in person some of the people I work with and talk to daily online was the icing on the cake!

At the beginning of October, we had three free online Nextflow and nf-core training sessions, in three different time zones, where 890+ people in 60 different countries signed up. What a success! I was responsible for the session in the Americas and I was very pleased with the result! Thanks to amazing people like Robert Syme and Ben Sherman, who simultaneously answered questions while I delivered the training, everything went super well! In the other time zones, I gave the same support they did to colleagues like Abhinav Sharma (it was morning in Brazil, so it was OK) and Chris Hakkaart (this training started at 00:00 in Brazil πŸ˜…).

Participants from Europe, Africa, and the Middle East (EMEA) particularly surprised me with their enthusiasm and participation! I answered more questions and solved more problems during these pieces of training than in the previous 2 months combined! And the training only lasted 3 days 🀯 In many ways they were milestones in my path in the community, like the hundredth technical question I answered on Slack that took place during the training πŸ€“

A few days after the training sessions, I was already in Barcelona again for the nf-core Hackathon where 50+ people from different countries participated in person and 230+ remotely. I led the documentation team with Abhinav Sharma and Franziska Bonath. What an experience! As expected, our team had a larger number of beginners, but that didn’t stop us from rolling up our sleeves and contributing πŸ₯³

During the three days of the Hackathon, the documentation team opened more than 50 Pull Requests, 48 of which were finalized and merged at the end. It was very cool to watch the integration of both in-person and remote attendees, not only among themselves but also with the other teams during the event. In fact, the participant with the most contributions from the documentation team was attending online! Below are some Hackathon statistics presented by Phil Ewels, Lead Developer Advocate at Seqera Labs and co-founder of the nf-core project.

General stats for all teams
Number of issues created on GitHub by author (I’m mribeirodantas)
Number of Pull Requests opened and / or merged by author (I’m mribeirodantas)

My task, as one of the team coordinators, in addition to managing the activity, was to find problems, create issues for them, help people solve them, and then review/merge the contributions. It was great to see people motivated, fixing the issues, and opening pull requests, so cool! Did I want to resolve some issues? Sure! But it was much nicer to see people who sometimes had never used Git, using it for the first time and managing to make contributions to an open-source project. From the plots, you can see that I was constantly opening/closing issues, reviewing and merging pull requests πŸ˜† Below are some photos from these amazing days πŸ₯³

By the end of the Hackathon, almost everyone had gathered in a cooking competition (hence the aprons!) sponsored by the Amazon AWS crew. Describing this activity does not do justice to the fun it was to participate in. A delight of activity 😬.

Culinary competition!

Do you think it stopped there?πŸ˜… No! The end of the Hackathon marked the start of the NextflowSummit 2022, also in Barcelona. Having witnessed the event unfolding as an idea back there, happening, and especially reflecting on everything that happened now that it has ended, is a unique experience! This event has gone down in history! I met amazing people, and the content was top-notch. Talks by representatives from Amazon, Google, Microsoft, Seqera Labs, Quilt Data, Illumina, and Nanopore, among other companies, were very enriching. The event was recorded and can be watched on YouTube by clicking here.

Opening talk with Dr. Evan Floden, CEO and co-founder of Seqera Labs.

The event was attended by hundreds of people in person, and many more online. You can check the schedule here and get a taste of what happened! In addition to participating in the organization, I was invited by Saba Nafees from Chan Zuckerberg Biohub to participate with her and Dr. Alison Meynert from the University of Edinburgh on a panel entitled “From sharing our journeys to empowering the community: Nextflow & beyond”.

Photo taken during the panel.
Moment of the panel where I spoke about the community in Latin America.

The panel can be watched on YouTube by clicking here.

During the Summit, there was a social night where we all went to a restaurant… Which was actually a nightclub 🀯 We had some starters at the beginning, sushi, very delicious dishes… And suddenly the tables were removed and gave way to the dance floor. It was at Shoko in Barcelona and as you can see from the image below, the decoration was astonishing 😻!

At the end of the event, while the participants went to rest, we at Seqera Labs embarked on a new adventure: the annual Seqera Labs retreat. The nf-core Hackathon and the Nextflow Summit were excellent opportunities for the company’s employees, spread across continents, to meet and get to know each other better, but that wasn’t the focus there. It was at the retreat, near Barcelona, that we were able to really integrate better and come together, through various activities, to reflect on the path that has been taken so far and how to achieve the goals we have for the coming years.

I had to cut several paragraphs from the original version of this post so it wouldn’t be too long to read, after all, believe me when I say that these past 30 days felt more like months! It was a very positive experience to get closer to colleagues I had only exchanged a few words online, and to celebrate in person with those I had already created a connection with, albeit online. All episodes of these 30 days were a lot of learning and I am very grateful to everyone who has been with me on this journey! Lots of learning!

And now? Fortunately, there was a lot of positive feedback after each of the events I mentioned in this post. We already have several activities in progress/planning that were influenced by the feedback we received, both after the panel and after each of the activities.

Are you not part of the Nextflow and nf-core community yet!? I don’t believe it! Join us in this community that is revolutionizing the use of workflows in the life sciences πŸ₯³. Click here to join Nextflow’s Slack and here to join nf-core’s Slack