Reevaluating the Tidyverse: Personal Insight and Future Directions
After eight years immersed in the Tidyverse, my recent experiences have prompted a reevaluation of its relevance and practicality. While I've always admired the work of Hadley Wickham and have appreciated Tidyverse's utility, my project with the University of Texas revealed specific ABI issues, particularly with the ‘httpuv’ package—a key component for both Tidyverse and Shiny functionalities. This acknowledgement isn't just an idle observation; it's an indication of potential pitfalls that can affect other users and projects relying on Tidyverse.
Dependency Challenges and Community Dynamics
Initially, my goal was to reduce the dependencies of ‘httpuv’ by utilizing R's C API in a way that some may consider outdated. Tackling dependency issues can lead to insights about not only R's ecosystem but also about how communities evolve in response to perceived innovation or obsolescence. The perceived pushback in public forums left me questioning the dynamics of communication within the community. In instances like this, constructive criticism or support can often be confused with hostility, resulting in a delicate balance of engagement.
For instance, I faced comments suggesting my contributions were off-mark, which felt disheartening. In an open-source environment, where contributors may often feel vulnerable, such remarks can dampen the collaborative spirit that initially drew many to this space. After all, feedback is essential, but it should encourage an atmosphere of improvement rather than dejection.
Efforts and Engagement in a Complex System
After dedicating considerable effort to a pull request (PR), I opted to create a second PR using Posit’s cpp11, yet it went unnoticed, even after I attempted to engage with contributors via GitHub and email. The inefficacy of my outreach highlighted a broader issue in community responsiveness. It isn't uncommon for submissions to languish in obscurity, especially when the number of maintainers is limited or if projects become bogged down by an influx of similar contributions.
Reflecting on my journey with R, starting in 2015 with data.table, I first encountered the Tidyverse as a promising tool, despite the initial struggles I faced with installation on Linux. Those early experiences shaped my perspective, as the familiarity gained with data.table showcased R's standard capabilities, achieving results without the abstraction layers the Tidyverse introduces.
The Divided R Community
This situation highlights an ongoing divergence within the R community. As I co-maintain the CRAN Web Technologies task view and actively engage in bug fixes and package maintenance, I'm reminded of the vibrant energy of a global community—yet the occasional negativity can taint that experience. It raises an important question: Is the R community becoming overly polarized between purists and those who embrace packages like Tidyverse? This division can often detract from collaborative potential across varying levels of expertise.
Having transitioned from viewing myself as a solitary contributor from Chile to now living in London while pursuing a PhD with a focus on R for Econometrics, I feel proud of my role in initiatives like R Para Ciencia de Datos. However, the sentiment around certain interactions remains a barrier to collaboration, contrasting sharply with the supportive relationships I’ve built over the years. Connecting with others in a positive manner yields better results, and while it's easy to pass off criticism as a personality clash, meaningful engagement is ultimately what promotes growth in a community.
Revisiting Base R
In a bid to streamline my workflow and return to the core elements of R, I've made the decision to remove the Tidyverse from my laptop entirely. By shedding this layer of abstraction, I hope to explore how to handle common data cleaning tasks with minimal typing using base R functions. Though I'm becoming a bit rusty with functions like apply() and mapply(), I've found that this shift will allow for greater flexibility and understanding. Retraining my focus on base R not only acts as a form of personal challenge but might also better align my skills with traditional programming paradigms.
Implications for Data Practitioners
What this means for you, the practitioner, is worth considering—especially if you find yourself wrapped in the Tidyverse bubble. There’s no denying the convenience and speed with which Tidyverse can handle data manipulation tasks. However, if you’re looking to deepen your engagement with the mechanics of R, returning to its foundational capabilities can provide valuable perspectives. And this is the part most people overlook: the act of ‘unlearning’ certain habits rooted in higher-level abstractions could lead to more profound insights.
As I document this transition, I aim to share insights that might resonate with others considering a similar path. While the Tidyverse certainly brings a set of well-known conveniences, my focus will now be on maximizing R's native capabilities to engage more deeply with data, especially in the realms of data validation and cleaning. This could also spark conversations about the varied experiences of users and foster an environment that encourages learning from both sides of the R divide.
Stay tuned for practical tips and methodologies that may help others unlearn certain Tidyverse habits while embracing the nuances of base R. As the discourse continues to evolve, the role of direct communication and respect among community members will remain pivotal in shaping the future of R programming.
