Renzo Francesco Lucioni
Director of Research Engineering at LogRocket. Interested in product development and data visualization.
Contact
- Web: https://renzolucioni.com
- Email: renzolucioni@gmail.com
- GitHub: https://github.com/rlucioni
Skills
- Languages: Python, JavaScript/Node.js, Bash
- Web Frameworks: Django, Express, Flask, React
- Ops: Kubernetes, Docker, Helm, Terraform, GCP, AWS
- Data Science/ML/AI: Jupyter, Pandas, BigQuery, Vertex AI, OpenAI, LLMs, embeddings
- Other: Pub/Sub, Bigtable, Postgres, Redis, Elasticsearch, GraphQL, OIDC, Git
Experience
Director of Research Engineering at LogRocket (July 2024 to present)
- Worked across teams to develop, deploy, and support LogRocket/Galileo Highlights, a way to accurately summarize user sessions individually and in aggregate, complete with inline links to key moments within sessions.
- Shipped a version of Highlights that uses user-submitted content from support tickets to guide summarization and discover relevant details of the user’s experience, intended to help support agents reduce ticket handle time by contextualizing users’ requests. Fixed long-standing issues with LogRocket’s Intercom and Zendesk integrations at the same time.
- Continued to improve issue classification and description capabilities, focused on improving precision without sacrificing too much recall. Started to explore multiclass classification to account for problems that don’t fit neatly into “severe” and “not severe” buckets (e.g., UI/UX issues).
- Managed LogRocket’s relationship with OpenAI, making sure we had spend and usage limits in place that could accommodate our growth.
- Encouraged eval-based (vs. vibes-based) development of new features, especially those using LLMs and similar technologies, across the product org.
- Investigated ways to build “funnel insights,” a product capable of explaining how users who don’t complete a conversion funnel behave instead, meant to inform things a customer might want to try in order to meaningfully improve conversion.
Senior Engineering Manager at LogRocket (September 2021 to July 2024)
- Advocated strongly for use of machine learning in LogRocket’s product, specifically to improve Issues. Our ML-based “Galileo” suite of features later became the most highly anticipated product launch in company history with over 500 customers on the waiting list.
- Established a small data science/ML team focused on developing an issue classification model that could be used to automatically recommend important issues to our customers, accelerating the triage process and increasing engagement with Issues.
- Led development of tools and processes needed to train and continuously improve machine learning models for predicting issue severity. This included collecting and preparing training data, feature engineering, getting familiar with Vertex AI, BigQuery, Jupyter, and Pandas, measuring model performance, and transforming raw model output into a metric that made sense in the context of the product.
- Validated model quality while helping the Proactive Insights team punch above their weight by using the model to deliver early, human-in-the-loop issue recommendations to customers (e.g., Proactive Insights, Issues Digest).
- Integrated the model with LogRocket’s production system in collaboration with the Issues engineering team. Our issue classification pipeline was capable of serving predictions for all LogRocket Pro customers (i.e., millions of sessions a day).
- Continued to spearhead work on Galileo by developing and deploying a method for using LLMs to generate natural language issue descriptions, reducing false positive issue classifications at the same time. This work helped grow Galileo into a collection of AI-based features that shaped our product roadmap, generated significant customer interest, and set us apart from our competitors.
- Authored a patent application covering our approach to automatic triage and description of software issues.
- Developed a “usage summaries” proof of concept demonstrating the feasibility of - and customer excitement around - using LLMs to query sessions (i.e., answer questions by summarizing sessions more generally).
- Broke ground on multimodal (text and image-based) session summarization, work that formed the basis of future Galileo features.
Engineering Manager at LogRocket (January 2021 to September 2021)
- Helped organize LogRocket’s engineering team into smaller teams focused on different parts of the product.
- Led and grew an engineering team. We delivered substantial new product to customers, including limited-lookback conditional recording, new Issue types, a major redesign of the LogRocket dashboard, the vertical event timeline in session replay, improved iframe recording support, and integrations with NPS tools.
- Acting as tech lead, shipped changes to improve performance and make our system more robust. Added incremental timeseries caching using Redis. Removed Kubernetes CPU limits unnecessarily throttling our containers. Removed the Cloudflare proxy from our event ingestion endpoint, preventing our SDK’s request rate from triggering Cloudflare’s DDoS protection and taking down this critical endpoint; simultaneously switched to use of Let’s Encrypt TLS certificates via cert-manager. Added a dedicated asset caching worker to help stabilize event processing. Significantly reduced an internal exception processing service’s Redis usage, improving stability and cutting costs by 3 orders of magnitude.
- Wrote handbooks for the rest of the engineering team to help spread siloed knowledge, specifically guides to the entire engineering interview process and handling ops-related incidents.
Lead Software Engineer at LogRocket (January 2020 to January 2021)
- Worked to scale LogRocket, keep costs under control, make our system more robust, expand our feature set, increase MRR, and grow our team
- Introduced dynamic data deletion policies to account for customers with shorter retention. Required developing a way to do fast streaming Bigtable deletes. Let us shed hundreds of tebibytes and tens of nodes.
- Simplified user activity tracking, replacing a complex and expensive Redis-backed buffering system with a simple event-based approach involving Elasticsearch
- Reduced cloud costs with GCP committed use discounts, E2 VMs, and AWS reservations
- Built support for backing up Bigtable, our 500+ TiB primary datastore, several times a day
- Made data ingestion cheaper by bypassing Pub/Sub where possible, and more reliable by adding separate ingestion pathways for high volume customers
- Stopped a Redis memory leak from taking down part of our system by finding large, previously-unknown, unused, and non-expiring Celery pidbox keys with RedisInsight, stopping them from being written, then unlinking them
- Identified SDK ingestion problems due to Cloudflare DDoS protection and negotiated with Cloudflare’s support team to mitigate
- Implemented a “path analysis” algorithm for building trees used to visualize overlapping user flows from many sessions
- Integrated LogRocket with New Relic APM distributed tracing, linking from network requests in LogRocket sessions to corresponding backend traces in New Relic
- Increased maximum supported data retention from 3 months to 2 years, an exercise in reevaluating long-standing assumptions - decoupling video and search data retention, being smarter about ES index use and data archival - and a useful price anchoring tool for the sales team
- Rooted out flaky tests to pave the way for requiring passing tests on PRs and Kodiak PR automerge
- Helped build a great team through referrals, active involvement in the hiring process, and willingness to teach new engineers about the system and our processes
- Promoted a culture of rigorous, valuable code review. Set an example of what it means to give a good code review.
Software Engineer at LogRocket (August 2017 to January 2020)
- Developed web applications using JavaScript and Python
- Improved JavaScript SDK performance and memory usage
- Built error reporting product. Initially called Errors, later became the more general Issues.
- Listed LogRocket on the GitHub Marketplace
- Reduced cloud spend by optimizing our use of Bigtable, Pub/Sub, GKE, and GCS
- Built support for on-prem (i.e., on customer-controlled infrastructure) deployment of the LogRocket backend using Terraform, Kubernetes, and Helm
- Secured enterprise contracts by developing, installing, and maintaining our on-prem/self-hosted offering, often working directly with customers
- Researched new product opportunities with Dataflow and Pandas, focused on finding user frustration
- Increased trial to conversion rate and ACV by planning, building, and shipping new products that became LogRocket’s Pro plan (e.g., Metrics, retroactive filters)
Senior Software Engineer at edX (July 2017 to August 2017)
- Implemented GoCD continuous delivery pipelines used to deploy daily
- Automated translation of application text using Jenkins and Transifex
Software Engineer at edX (July 2014 to July 2017)
- Developed open-source web applications using Python and JavaScript
- Built ecommerce service handling millions of orders
- Maintained services supporting edX Programs (grouped courses)
- Scaled metadata service underpinning edX products
- Load tested new applications and features with Locust
Freelancer at The Economist (December 2013 to November 2014)
- Created data visualizations for use by the paper, online and in print
Intern at edX (May 2013 to June 2014)
- Implemented inline problem scoring
- Built lightweight A/B testing framework using Django Waffle
- Instrumented platform with Google Analytics and Mixpanel
Education
Harvard University (September 2010 to May 2014)
Bachelor of Arts in Computer Science, secondary field in Government