Huddle Agenda 2019
How Does It Work? Simple! Each conference delegate participates in seven huddles (discussions) each – two on day one and day two, and three on day three of the conference. Each huddle lasts for 90 minutes. There are over 50 huddles to pick from, split into seven time slots (round #1 to #7).
Register now and we will let you know once huddle selections are open (expected in July).
Huddles Day 1 - September 9, 2019
Key Performance Indicators measure an organization’s progress against specific business outcomes and are often linked to revenue goals. For mission-based organizations, simple economic indicators don’t measure the real success of an organization in achieving its mission. How do you set meaningful KPIs when the outcomes are inherently difficult to measure?
In this huddle, Amy Sample from PBS will lead a discussion exploring the processes organizations take to set KPIs. We will explore the differences between for-profit businesses and mission-based organizations. What can we learn from each other about how to set actionable metrics that connect to strategy? How do we build organizational buy-in for the metrics we recommend?
We will also explore the challenges inherent with measuring mission-based outcomes like literacy improvement, civic engagement, or curing disease. Join us for a stimulating conversation about measuring success.
For many organizations, big data is an untapped resource of intelligence that can support business decisions and help provide customers a better service. As data volume and types continues to increase and change, more and more organizations are embracing predictive analytics, to take advantage of data at scale.
In this huddle, led by Gary Angel, CEO of Digital Mortar, we will share successful use cases of digital data-based predictive modelling.
We will then dive deeper into questions such as:
- Which business challenges lend themselves to predictive analytics
- What is the role of machine learning in solving these problems
- How can we move predictive modelling from the world of the mathematician into the world of the business analyst
You will come away from this huddle with practical examples and a better sense of direction on how to utilize predictive modelling in the digital environment.
For three years running, Glassdoor has ranked "data scientist" as the #1 job in America. High pay, high satisfaction, and high demand. But what even does it mean? Let's cut through the hype and talk about the practical aspects of both the analyst and scientist roles in 2019.
June Dershewitz’s team in Twitch includes both analysts and scientists. In this huddle, June will navigate us through topics and questions such as:
- What’s in a name? Is the role of the analyst shifting towards the data scientist to the point we can use the latter title to attract more candidates?
- Leveling guides, skill sets, deliverables, career pathways – what are the distinctions between different data-related roles, and how can people transition between them?
- Organizational models – where are the data analysts, where are the data scientists? How do they work together most effectively and how do they route work to one another?
This huddle is open to both individual contributors and managers. Attend this session if you see your role changing and are looking to understand what this change means to you or if you are a manager trying to figure out what is the right mix of data staff and how best to utilize each skillset.
You have grown from one part-time A/B testing resource to an optimization program of nine team members running both quantitative and qualitative testing. You had lots of growing pains! You manage a governance team which sets your roadmap and priorities. And yet you feel that optimization is not delivering to its full potential.
Time permitting, we will touch upon the role of agile and a product-centric approaches to delivering ‘lean learnings’.
This is exactly the challenge Paula Sappington of Hilton faced. In this huddle we will examine how to drive more out of our optimization programmes. We will look at examples of effective prioritization processes, how to measure your program’s health, the role of analytics and executive reporting and scaling the program up to multiple product groups in a timely manner.
The most important part of a track and field relay race is the handoff of the baton to the next player. How many great athletes have been foiled in the final seconds by an epic failure of the baton handoff?
Great analysis follows similar paths when a presentation of the work is not understood by stakeholders and the insight lost. We would argue the delivery of the report is as important as the analysis itself.
In this huddle, we will be focused on the following challenges:
- How to say 'no' to work that does not provide business value
- Building the 'brand' of your analytics team to garner an audience
- How to become and remain the neutral, yet trusted 3rd party in the room
- Tactics on how to distill complexities of the data into business insights focused on driving action
- How to 'own the narrative' of the story and not let your analysis take on a 'life of its own'
- Should communication skills be table stakes for hiring a good analyst?
Please join Matt Wolf of Dignity Health in this discussion as he draws on nearly fifteen years of experience from Nike and Sprinklr as a practitioner and leader of analytics, data science and data governance.
Digital analytics teams continue to expand, develop deeper specialization (SEM/SEO, Visualization, Content Optimization, Tag Management Testing/Optimization) and grow new roles and responsibilities (Audience Development, Management, Segmentation, Personalization, Campaign Automation).
In this huddle, Rusty Rahmer of Vanguard will lead a discussion on the emerging best practices and organizational models for organizing your analytics team around these activities, servicing your organization, and techniques for developing the talent on your team to meet the demands, and achieving organizational alignment with partners.
Join us for a conversation that is guaranteed to generate exciting insights about analytics team organizational models.
If you work in an organization that was formed prior to the ‘digital revolution,’ chances are that your digital presence has grown as an extension of your ‘main’ business. In many cases, this means that both the digital properties of your organization and the analytics implementations on those properties are a patchwork of disparate efforts and a wonderful monument to a bygone era.
Join Rastko Kovacevic of Delta Air Lines in this huddle, which will explore the ways in which a legacy analytics implementation can be transformed to support the needs of a modern digital business. We will explore the following:
- How to get organizational buy-in to invest the time and resources in a large, time-consuming project
- Which is better – tackling the implementation problems piecemeal or a wholesale ‘teardown and rebuild’
- Is this something that can be done in-house or do you need external help
- Does the organizational structure need to change as part of this transformation
- Are the existing tools and processes adequate or do they need to be updated
- What are the new topics, metrics, and insights that become possible post-transformation
- How do you prepare for new ways of digital interaction (voice, etc)
Product development, from feature specification, to engineering and deployment is rarely straightforward. Adding requirements to A/B test features will seem like an extra burden.
In this huddle, led by Anthony Tang, Senior Engineering Manager at Walmart Labs, we will discuss how experimentation should be assimilated into the product development lifecycle, and how to turn it into a benefit, not a burden. What is the value of A/B testing in developing new features? How can an organization add split testing to the product lifecycle? And how is engineering involved in implementing the test variation?
Let’s talk about how experimentation can be integrated into the product development lifecycle so that the impact of features released can be quantified, quality can be improved, and your experimentation ROI improved.
Multi-touch attribution is often framed as marketing analytics nirvana. Numerous software vendors and consulting companies offer plug and play attribution tools. So why do many companies still use rule-based attribution models like last touch?
Implementing new attribution models at the enterprise level comes with a unique set of challenges. The two most common reasons for the failure of organizations to adopt multi-touch attribution are:
- Lack of executive buy-in
- The (physical) disconnect between the attribution tools and the marketing tools used to execute campaigns
Sarah DeAtley focus in Microsoft is marketing analytics. In this huddle, she will stimulate delegates to share their attribution modelling experiences. We will look at how to explore new attribution models without wasting time and resources on solutions that will not be adopted? Sarah will also cover some of the most common pitfalls of scaling advanced attribution modelling in an enterprise company and how to avoid them.
Expect to come away with vital understanding to help you thrust you forward in your attribution journey.
Baby Boomers are emotionally invested in their company’s success and are very loyal. More than 40% of Baby Boomers stayed at one company for the majority of their career. Millennials, however, are much more fluid. They prioritize experiences over steady linear careers, switching jobs every 1 – 2 years. Many of them harbour an inherent distrust of corporations.
But it is not just a generational difference. Society has shifted, fuelling the fire of this change. For example, pension plans and healthcare benefits are evaporating, reducing the incentive for long-term tenure. Long-term benefits are being replaced with the immediate gratification of Keg Thursday and Friday Mani-Cube.
In this huddle, led by Laura Beaulieu of Talbots, we will discuss:
- What does that shift mean for hiring managers?
- What motivates the Millennial?
- What should the career ladder look like?
- How do managers compete with the 1 – 2 year job hopping cycle and sustain promotions every 1 – 2 years?
There is a shift in team design and sustainability – in this huddle we will crack the code of “The Millennial” when fostering our analytics team. We welcome both millennials and managers to a huddle which will offer great benefit for both.
Governments and nonprofit organizations often have KPIs that are not directly tied to increase in revenue. Most of the machine learning (ML) and artificial intelligence (AI) trainings, examples and success stories involve targeted ad placement, image recognition or recommendation engines. Yet not all activities neatly fit into these categories. Government sites are often information dissemination sites without ad placement and nonprofit organization (apart from the donation funnel) are informational in nature, supporting off-line fundraising activities. So is ML/AI out of limits for the noncommercial sector?
Fred Smith, Technology Team Lead and Digital Metrics Lead at the Centers for Disease Control and Prevention (CDC) will refute that suggestion. Leaning on his extensive experience both in government and ML/AL, he will lead an interactive huddle to discuss ways of using ML/AI to generate key insights from various digital metrics and resources to achieve a variety of mission objectives. We will discuss how ML/AI could be deployed for a variety of applications including predicting and reducing bounce rates, search engine optimization, non-revenue generation calls-to-action conversion funnels and even content recommendations and more will all be discussed.
This huddle is open to delegates from all sectors and will offer benefits beyond just the governmental and nonprofit sector. By the way, our model show that you have a 38.23% chance of joining this huddle and 100% of coming away with tangible outcomes.
Setting up mobile media measurement for mobile apps is a complicated process. It requires a complex data architecture using mobile marketing software and measurement tools. Once all systems are ready, you then need to establish what metrics you want to focus on to measure your marketing efforts.
In this huddle, Brian Hudnall of Starbucks will lead a discussion on the metrics, processes, and tools that we should be using for mobile media measurement. We will also examine how mobile media metrics align to site metrics.
If your mobile app tracking strategy needs a reboot or just a recharge then this huddle is for you. You will come away with new ideas to boost your mobile app analytics prowess.
The retail landscape is undergoing a revolution in recent year. Much of the change is driven by an increase in consumer needs as well as shortening product and trend life cycles. Retailers are struggling to keep up. Digital data plays an important part in helping retailers overcome the challenge. But on its own, only offers one dimension of the whole picture.
In this discussion, led by Yuriko Medina-Frank of Goldman Sachs, we will examine how to maximize the value of digital data and the enrichment of data sources for better merchandizing. This will be a valuable and exciting session for those looking to discuss:
- Customer journey analytics through device stitching (digital and direct mail impressions to conversion)
- Site to store (online browsing to store conversions and every variant in between)
- Store and visual merchandising through basket analysis and regional analysis
- Site visual merchandising and recommendations
- Site creative asset placement for personalization
- Product feedback loops (leveraging reviews and return data to provide quantitative and qualitative feedback on product performance)
You should expect to come away with fresh ideas on how to improve your product and merchandizing analytics.
Data & Insight teams are critical at any large organization. Reports, dashboards and insights produced by these teams help the business better understand customer behaviors and support decision making. Nevertheless, even with the best tools, great analysts and smart KPI’s we often struggle to drive engagement.
Rather than blame the business let us discuss and share methods to improve engagement with the non-analyst part of our organization. We will discuss:
- Effective data storytelling use cases
- What are the different approaches and techniques that drive stakeholder engagement? How much is subtle improvements in data visualization compared to insights narrative?
- What is the right balance between data preparation vs story telling? How do we measure success?
- Do analytics teams have the required data storytelling skills? Where are the gaps and how can those be overcomed?
Come share your experiences and views so together we can tackle one of the fundamental challenges we all face in analytics.
Huddles Day 2 - September 10, 2019
A/B Testing is a powerful tool that allows product management, developers, and the HiPP of HiPPO fame to measure and decide what to release. However, can we run an effective experimentation program without becoming a bottleneck?
In this huddle, led by Anthony Tang of Walmart Labs, we will discuss different approaches to managing the A/B testing program in your organization:
- Is it better to have a centralized testing team?
- Should A/B testing be open to anyone to start their own tests?
- Are there any hybrid structures that could lead to better experimentation results?
There is no one-size-fits-all answer to this challenge, however, the goal is the same – to run an efficient experimentation program. Join us to share and discover how we can all achieve that goal.
Since our last conversation, the Women in Analytics movement has marched on. 2018 has been the year of the woman around the world. In the analytics world, Columbus, Ohio hosted the Women in Analytics conference, the DAA launched a formal mentoring program, and networking groups are popping up in organizations across the industry.
Let’s continue our conversation. What success stories do you have to share with the group? What techniques have you applied in your organization to attract and retain more women analysts? How do we address the differences in equity perceptions among men and women? How do we achieve a better gender balance in senior leadership positions? Where do we need to focus our energies for the next year?
We welcome both women and men for what continues to be an exciting discussion about our profession.
Widespread interest and adoption of Machine Learning (ML) is bringing significant change to analytics processes. In this Huddle, led by Gary Angel, we will discuss what you need in order to do ML and how it changes the analytics approach.
Key topics will include the requirements for supervised learning data sets, data transformation and construction techniques, the pros and cons of working with lots of features, challenges in multi-label analysis, problems in organizational verification and transparency of model building, and techniques for moving from data science exploration to operationalizing models.
Maybe it is a new user or a returning one using a new browser. Maybe you are not allowed to set persistent cookies for privacy reasons. Either way, how can we make refined predictions on who is visiting our sites quickly in order to improve stickiness or to achieve our call-to-action events?
What about a scenario where you do know the user and information seeking habits but user tasks change according to current user role? For example, a person seeking health information for themselves today, their elderly parents tomorrow and their young children at 2 am? When, if anytime, is ‘not knowing’ advantageous over knowing? When does good search engine ranking work to your disadvantage?
Fred Smith of the Centers for Disease Control and Prevention (CDC) will lead a discussion on the various ways that user roles can be predicted and the various data points that may be relevant for such predictions. Fred will share what CDC is doing and planning to do to improve visitor experience with CDC.gov by predicting visitor roles, recommending relevant content and rethinking search engine optimization strategies.
Come share your experiences and challenges – you will leave with fresh ideas of how to exploit user roles for both content and conversion.
Businesses are increasing spend on programs to help improve customer retention and loyalty. Data is critical in driving success to personalized customer experiences and targeted campaigns. To do so you must get your analytics strategy right and data structures in place to support such programs and help reduce churn.
In this huddle, led by Sid Shah, Head of Analytics & Insight at Conde Nast International, we will discuss:
- How to leverage existing data to measure customer loyalty and understand their behaviour?
- Which tools, techniques and analytics strategies should you use to support uplift in customer retention?
- How customer data can be used to build and optimize great products?
Share and explore different analytics practices that will help us all build or improve our customer retention and loyalty programs
Large scale digital experiments that have an impact on various aspects of an organization’s business require, in most case, the participation of multiple business groups including finance, technology, marketing, etc. This demands a robust experimentation program to clearly set expectations on strategy, methodology, and technology requirements.
In this huddle, Brian Hudnall of Starbucks will discuss challenges and learnings that the Analytics & Insights team has experienced in building their own experimentation program within a cross-functional organization. We will discuss the importance of teaching the value of experimentation, program management, measurement plans, data architecture, and clear story telling for a cross-functional audience.
Come learn and share experiences from your experimentation program scaling efforts.
Most organizations do not have an abundance of data scientists that can help ingest, process, and make sense of the vast amounts of data that is potentially available to them. Even with ever more advanced machine learning and artificial intelligence products, it can be difficult to get at the highly insightful data nuggets that can help inform great business decisions.
Rastko Kovacevic of Delta Air Lines will lead this huddle that will discuss a topic very near and dear to his heart – perfecting the design and collection of digital data so that it can easily answer business questions and generate insights. We will cover:
- How do the primary digital KPIs get defined?
- Who participates in that process?
- What is the process of designing tracking to obtain the best possible measurement?
- How do you pick what to track?
- Why tracking everything may not be the best idea?
- How can you maximize insights while minimizing costs of server calls, data storage, and implementation?
- What is the role of the business product owners, analytics, and implementation teams in putting all this together? How do they best interact?
- What is the best way to establish a business process for this type of workflow?
- How do you best document everything to streamline improvements and future iterations?
Come share your experiences and opinions and enjoy a rich exchange of ideas.
Tom Betts was the first board-level Chief Data Officer at the Financial Times in 2015. Join him to hear why the FT made the CDO a board level position and to discuss how data can connect your strategy - and the desires of your executive leaders - to the work on the ground. As the evolution of technology accelerates and businesses push for cross-discipline collaboration and greater decentralization the requirement for a common language is critical to ensure that disparate teams and departments remain aligned, pulling in a common direction. How do businesses go beyond the design of good KPIs, to embedding them in organizational processes and our day to day workflow?
- How do you decompose your organizational strategy into metrics that matter? Or even better, one metric that matters.
- How do you embed an experimental mindset in areas of the business where this does not come naturally?
- What techniques work to develop a data-rich culture in your organization, and incentivise teams to use data to improve their work?
- What are the tools and techniques that organizations can use to incentivise the use of data and metrics?
- KPIs, OKRs, and more… what techniques best help align teams around common goals?
If you are eager to understand how to really make your analytics impactful, valued throughout your organization and a key driver of growth then join this huddle.
Technology is enabling greater access to data but it does not, on its own, enable stakeholders to self-serve. Technology must be partnered with data management and visualization strategies that make it easy for users to consume data easily and confidently.
In this huddle we will explore the BI tech landscape and the supporting data initiatives that lead to successful deployments in our organizations. Led by June Dershewitz, Director of Analytics at Twitch, we will look to answer the following questions:
- What data visualization tools are you using and why? Have you got an intentional portfolio of enterprise BI tools, or a mish-mash?
- What kinds of tools, systems, and strategies are you exploring for the future?
- What are the best and worst things happening in your business with respect to data visualization? Let's share successes and war stories.
- How are you managing governance of reports and data sources available in your BI tools? Lessons learned? Tips and tricks?
Attend this session if your area of oversight includes BI, or if you are an individual with strong point of view about data visualization tools.
The growing maturity across leading experimentation programs is causing a shift in the industry landscape. Experimentation professionals are growing frustrated with use cases where common client-side split testing cannot deliver a viable solution to their testing needs. For example, testing product functionality, including new features, backend logic, algorithms etc.
Enter server-side testing. In this huddle, led by Paula Sappington, Sr. Director, Digital Optimization & Experience Design Research at Hilton, we will consider some emerging use cases for server-side experimentation and the many associated challenges.
We will also look at the following questions:
- When should you start considering the introduction of server-side testing?
- Should you combine client-side with server-side testing? How best to do that?
- How does server-side testing impact your experimentation team (personal & process)?
- Does server-side testing require additional changes within your business
You should expect to leave the huddle with newfound understandings about server-side testing which in turn will help ensure you are well informed when making your own server-side testing decisions.
Are you solving the need for analyst resources, talent, and tools but finding your organization is still struggling to achieve its full analytic potential? Has the greater opportunity shifted from the analytics team, to the non-analyst, marketers, leaders, and internal partners that surround them?
Rusty Rahmer, Head of Enterprise Web Analytics & Digital Intelligence at Vanguard, has experienced this shift in recent years. In this session we will bring attendees from internal analytics teams and external consultants together for a discussion around the challenges of offering strategic consulting from inside your organization versus outside, and vice versa! We will seek to assemble a list of the best practices, tips, and tricks that will help both sides be more successful and effective!
Choo! Chooo! The CDP train is pulling out of the station. The destination is yet to be determined but you better hop on! CDPs are like fidget spinners – nobody knows what they are for or why they want one but somehow fidget spinners became a half billion dollar industry overnight.
We can all appreciate the potential long-term value of a CDP. But what about the short-term? Implementing a CDP is a huge up-front expense plus a significant resource investment and time commitment. Any smart business will expect a quick return on that investment.
In this huddle, led by Laura Beaulieu of Talbots, we will discuss:
- What characteristics are important for an ideal CDP use case?
- What should you consider when selecting and implementing a CDP?
- What are some of the projects that will deliver that instant ROI from a CDP?
- What else needs to happen for your CDP experience to be a resounding success?
Join this huddle and walk away with a list of ideas to demonstrate value add immediately following a CDP implementation.
One of the most dreaded questions a marketing analyst can get is “can you measure the impact of this awareness campaign?” Marketers sometimes assume that just because a campaign is digital or has some level of tracking that it can be measured in the same way as a direct response campaign.
Upper funnel campaigns targeting awareness are difficult to measure—and can be very costly. What KPI do you choose to measure awareness? How long does it take to change brand perception? What are your tracking options when the campaign is not digital?
Sarah DeAtley of Microsoft will discuss tips and tricks for honing in on a measurement plan for brand campaigns, and when it makes sense to use digital behavior data.
In a world with an increasing appetite for data driven strategies, it is easy to get stuck in a rut of delivering metrics and reporting. Unfortunately, without the story behind the data, the metrics are at risk of being unused or worse… misinterpreted.
In this huddle, led by HCA Healthcare’s Analytics Team Lead, Ashleigh Bunn, we will roadmap the journey from metrics to analytical insight. We will:
- Explore the importance of change management respectfully pushing back on report requests for mere reporting sake
- Discuss approaches in eliciting the ‘real’ ask – asking the right questions to determine the impact your analysis will have
- Ask how to become a Data Partner rather than a Data Server
- Debate how best to develop a cadence of analysis
Join Ashleigh for an exciting discussion about one of analytics’ core challenges – the power to impact change through data and insight.
How do I 'Netflix' my data environment so my data scientists can do meaningful work? The first complaint of a data scientist is not having data, and the second is not having an environment that hosts their experiments and Machine Learning algorithms. Securing funding to build a data science architecture and knowing the right architecture to build is a herculean effort. Even if the funding is acquired, many big data programs fail from ancient architecture, scope creep, data governance and escalating cost and time to value, ultimately ending up in a lack of trust.
In this huddle, led by Dignity Health’s Head of Analytics, Matt Wolf, we will be looking to answer the following questions to help inform you in building a big data stack to maximize time to value:
- Hadoop, R, Python, ML, Data Viz, etc… how the f#$% do I make this work?
- Why will data governance and privacy policies make or break my program?
- How to make InfoSec, Legal and Compliance my best friends?
- What does an advanced analytics and data science team needs?
- What is the 'theory of data science' and why is that an important quality in a data scientist?
- My data scientist and data engineer think each other are aliens. How to bridge the gap between the two?
- How do I keep my data scientist from quitting because they are not doing data science?
We welcome both managers and practitioners to a discussion promising to be of real interest to both groups. Come away with a clearer view of what a successful data science environment looks like.
As the saying goes, the only thing that is constant is change. If digital analytics implementation projects where once perceived as ‘one-offs’, these days savvy organizations understand that they cannot afford standing still with their analytics implementations.
Even the most sophisticated analytics implementation must adjust to changing business requirements, a growing appetite for customer journey data and shifts in technology.
In this huddle, Yuriko Medina-Frank of Goldman Sachs will discuss how to evolve your implementation beyond the standard into the realm of deeper insight. We will also consider how implementations should help drive action and real-time targeting. Topics will include:
- Measuring price sensitivity through Product data and micro conversions
- Incorporating cost data as a tie breaker in product sequencing and recommendations
- Branching customer journeys through decision engines, even in SPAs
- Enriching and ensuring the right variables are captured
Bring your advanced analytics implementation examples and hear what other leading organizations are doing to keep their analytics framework fresh.
The explosion of available data is both a blessing and a curse. Safe to say that very few organizations are taking full advantage of the swelling data available to them. Most analytics teams are struggling to connect all available data sources, impeding them from having a comprehensive view of the customer. And with the advent of tighter privacy laws, additional consideration is required.
Autodesk has realized significant benefits from integrating diverse data sets, both digital and offline, to enhance and facilitate advanced analytics and modeling, segment its customer base and improve its testing capabilities.
In this huddle, led by Prolet Miteva of Autodesk, we will go through our development stages and emerge with better understanding of the possibilities and challenges of connecting and using customer data. Our discussions will focus on:
- Top data types to integrate (CRM, marketing, clickstream, inventory, competitive, social etc.)
- Value adds from connecting different data types (stronger models and more descriptive segmentation for marketing; context data for relevancy and personalization; optimizing for LTV and whole journey vs. one touch or campaign etc.)
- Data challenges (different keys, data integrity, closed platforms, inconsistent or changing data or formats etc.)
- Skill sets needed, tools and platforms used, new data integration process basics
- Challenges raising from increased privacy legislation
If merging digital analytics into other data sets is something you already do or want to do better then this huddle is for you.
Huddles Day 3 - September 11, 2019
It’s no surprise that as digital platforms proliferate, and a new streaming service is seemingly announced every week, the challenges around measurement continue to grow. Which constitutes the more pressing concern in this constantly evolving landscape:
- Ensuring that you’re accurately reporting across all of the channels that your content appears on (including linear TV, subscription video services, direct to consumer offerings, social media, mobile apps, OTT platforms and more)?
- Translating cross channel data into insights for management?
- Making certain that your measurement strategy is supported by the proper technological infrastructure?
What if it’s all three?
In this huddle, Joe Miscavige of PBS, will facilitate a discussion surrounding:
- What are the biggest nonlinear measurement challenges you face, and have they changed over time?
- How do you adapt to the changing media environment - does it require an entirely new measurement strategy or simply updating an existing one?
- What is the tolerance in your organization for analytical experimentation, whether around measurement strategies or new measurement technology?
- What challenges do you face in terms of implementation and do they affect strategic decisions you make?
While this discussion is primarily for those dealing with nonlinear measurement in a media environment, anyone who is currently grappling with a changing measurement environment or shifting measurement strategy will find this huddle useful.
We are told that every company is now a data company, and yet we wrestle with a variety of problems related to getting business people to value, understand, and use data in their day-to-day work.
In this huddle we will explore the topic of data culture and discuss what self-service means in the context of our own businesses. Led by June Dershewitz, Director of Analytics at Twitch, we will look to answer the following questions:
- How do you develop a strong data culture? Is it a top-down mandate from executives, a grass-roots campaign led by data staff, or a bit of both? What tactics actually work to build and sustain a data-driven mindset?
- Who are the different customers of self-service analytics, what motivates them, how do their needs differ, and how can you best enable self-service for each group?
- What are the dangers of self-service? What could possibly go wrong once you empower everyone to become their own analyst? Or, to put a positive spin on it, how do we make data literacy pervasive?
If your analytics resources are overwhelmed, you are wrestling with your organization over the adoption of self-service analytics or you are in data-driven nirvana then this is your opportunity to share and learn how to maximize analytics ROI via self-service.
Europe had the GDPR and has ePrivacy coming soon, US has CCPA (coming soon), and China recently strengthened its privacy legislation. Different jurisdictions have different rules, but they're fluid and changing. At the same time, browser technology is evolving to reduce the scope and scale of tracking. What does this all mean for digital analytics and digital marketing? What does it mean today, and what might it mean tomorrow?
- How are these changes impacting the digital measurement industry and how are you preparing for it?
- How are you ensuring the ethical use of data in accordance with your policies?
- Will our technology need evolve - to improve transparency in how data is used and to facilitate new freedoms
- How transparent do we need to be, and how easily will our tools allow us to be transparent - in what data we hold, how we process it, and how we're using it to make automated decisions about our customers?
- What opportunities might this new regime offer brands who transact directly with customers? If there is one emerging topic to know more about, this is it. Join Tom for an essential discussion about data collection, processing and privacy. You should come away with a clearer understanding of the increasingly critical topic of customer privacy.
It is fairly well understood how to split traffic and serve one set of users a control and another set a variation. But what if splitting the users is not an option? For example, testing how changes impact SEO rankings; or testing one type of change on a given set of products vs. another, where we cannot split the traffic without causing unintended bias effects.
In this huddle, we will discuss why such tests are sometimes necessary and the different techniques to run them. We will also answer the following questions:
- What do we need to consider when choosing groups of entities to compare?
- How can we analyze the results?
- How do we convey results outside of the experimentation team?
Note: this type of testing can potentially also apply to offline tests, among other situations. We will explore potential use cases and how to best manage these types of tests effectively to get meaningful results.
In 2006 Apple changed everything with the iPhone and now in 2019 our digital ecosystem has grown to over 17Bn connected devices globally. With the mass acceleration of data creation, storage and usage the conversation has shifted to a 'trust market'. Every enterprise serious about data collection is nervous about broad sweeping legislation likely to hit the US in the wake of Europe's GDPR. If your enterprise does not have data privacy and governance at the core of your strategic data platform conversations then you will likely fall victim to major litigation or an extremely expensive re-architecture in the future; both of which are paralyzing.
In this huddle, led by Matt Wolf of Dignity Health, we will discuss how to keep 'Data Privacy by Design' at the core of your data architecture with the following topics:
- GDPR, CCPA, ITP; why are these 'indicator laws' I should pay close attention to?
- What is a 'trust market' and why are marketing leaders talking about it?
- Are there Data Privacy by Design concepts I can implement today?
- Building a successful Data Governance Office and data ownership program
- Is the cookie dying a slow death?
- How to say ‘yes’ before ‘no’ when building a data as a service model
- Why should my best friends be in InfoSec, Legal and Compliance?
If data privacy and governance are key topics on your agenda then this huddle is for you. Expect to come away with tangible outcomes you could apply once back at your office.
Having a central analytics team within an organization is extremely helpful to ensure a consistent standard of analysis. For members of the team to gain the full value of the team environment there need to be tools and processes in place that promote sharing and feedback.
In this huddle, Brian Hudnall of Starbucks will moderate a discussion about the tools and processes that required for code/methodology review, and final product sharing.
- Consulting with partners that request analysis to clearly understand their questions.
- How to run an effective and efficient code review.
- What tools (Github, Databricks, etc.) to use for a data storing/updating.
- Using a central location for data product storage where partners can go to find final output.
If you strive to build trust in an elaborate data environment – this session is for you.
We spend significant time and effort justifying growing our teams. A growing team can mean growing success or growing pain depending on the team members’ characters, skills and the workflows in place. It is no longer realistic to look for the perfect unicorn. But the good news is you might not need to.
Join this huddle, led by Prolet Miteva, Senior Product Manager of Analytics at Autodesk, were we will share and discuss how analytics managers are maximizing the value of their team including:
- The pros and cons of growing teams
- How to pivot your team from small and multi-functional to large and specialist-focused?
- What process adjustments are required as your team expands?
- How best to nurture talent in an increasingly divergent team?
We invite both analysts and managers to join the conversation which will produce new insights for all.
Most of my DA Hub conversations are relentlessly tactical in focus- and that’s a good thing. But I think it might be interesting to have a different kind of conversation. I’d like to kick-back a bit and explore what we might do with our analytics teams that might be dangerous or interesting or transformative. Should we be deconstructing product? Investigating consumer cognitive psychology? Analyzing our own budgets? Gamifying optimization programs? If you’ve ever had a wild idea that you knew would never get budget or that seemed too hard or too radical to tackle, I (and I think others) would love to hear it and kick it around. If this sounds like a risky, unlikely conversation to be productive…all the better.
Technology is enabling greater access to data but it does not, on its own, enable stakeholders to self-serve. Technology must be partnered with a reporting and visualization strategy that makes it easy for the user to consume and be reinforced with ongoing education.
In this huddle, Mia Vallo VP Analytics at National Geographic Partners, will invite delegates to share experiences and opinions on their data democratization efforts. We will ask:
- Who in the organization should have access to our data?
- What is the balance between access to data and overwhelming with data?
- What makes a successful self-service analytics program?
- Is having a KPI framework for essential (allowing stakeholders to better understand the data and which KPIs they should be focused on so they can align their work with defined business goals)?
- How do we control misuse of data (e.g. for internal political usage)?
- What makes a good literacy program, helping stakeholders consume data and be reinforced with ongoing education?
- How should your reporting and visualization strategy look like?
You will come away with fresh perspectives on how to improve data share and utilization in your organization.
From being in the content production sector such as news and government sites, to relying on marketing and engagement content to increase sales revenue, content is king. Text, video, images, social media comments all support our mission and organizational goals. But how are we measuring the effectiveness of the content, evaluating potential improvements, know what content is contributing to our KPIs?
Or is the question itself a rabbit hole designed to lead to unproductive Brownian motion? Is “optimizing content for engagement and consumption” an impossible goal akin to the Heraclitus* statement that “you cannot step in the same river twice” or knowing which half of John Wanamaker’s advertising money is being wasted.**
In this huddle, Fred Smith of CDC’s Division of Public Affairs will lead what promises to be an interesting, and even possibly heretical, discussion on content analysis. We will discuss tools and approaches to content analysis through direct or indirect means. We will also debate when to measure and just as importantly when not to measure. How do we recognize the limit to our control over how our content performs and what can we do to potentially change that? Can machine learning for example shed light into this abyss? (Hey, I do not know either, but I have got some ideas and really want to hear yours).
All participants in this huddle will receive free health content to use on your site!
*yeah, I had to look that up. Who the heck remembers Heraclitus? Who can pronounce it?
** yeah, had to look that one up too!
Success is often found in taking risks. However, risks need not be uncalculated if couched within the context of experimentation. In this manner, risk is measured, anticipated and even hypothesized. Most organizations have testing ‘programs’. But it is those orgs that have encouraged, empowered and cultivated a culture of testing that are finding the greatest, most impactful and sustained success.
In this session, led by Ashleigh Bunn of HCA Healthcare, we will discuss some of the most successful practices that support a culture of testing within an org including:
- Developing an efficient testing process
- Propose, Hypothesize, Approve
- Recognizing when a test is not REALLY a test
- Empower your org (not just your team)…
- Engaging UX/UI and Integrating IT into the process
- Building a Testing Board of Directors
- Leveraging volume (even when low)
- Building a test repository/library to avoid test duplication and time wasting
- Learning from your failure and living to test again….
Most testing programs underdeliver not for lack of testing ideas, creativity, skills or stakeholders’ desire but rather for lack of a testing culture and a robust process to ensure volume, continuity and efficiency. Come share your success stories and challenges along the way. You will come away with practical ideas to improve your experimentation program.
Whether it is the scale or customization needed, at Microsoft, we often have trouble using industry standard products. However, this issue is certainly not limited to large organizations only. Are you struggling too?
Sarah DeAtley first worked at Microsoft on a new implementation of Omniture ten years ago. Eight years later Microsoft had since stopped using Omniture, moved to a new tool, built an in-house tracking system, and finally completed a “new” implementation of Adobe Analytics. Alternating between analytics solutions and wrestling with “build vs. buy” is a common exercise. In this latest case for Microsoft—we took a uniquely hybrid approach that involved our own tracking, Adobe Analytics as a visualization layer, and managing all digital data in our own big data stores. Microsoft is two years into this hybrid approach and there have been many learnings as an organization around scaling reporting.
In this huddle, Sarah will share and explore options of in-house analytics stacks. We will look to answer:
- How far can we go with industry standard analytics products?
- How do you obtain management buy-in?
- Do you have the in-house technical capacity to handle such a project?
- What are the potential pitfalls you must consider before embarking on this journey?
- How in-house analytics solutions will continue to evolve.
Whether you at the consideration stage, already integrating standard and in-house solutions or in post roll out, this huddle will help shape your thoughts around next steps.
As a discipline User Experience (UX) is focused on understanding your customers’ needs and delivering the easiest possible series of actions to complete an action. Traditionally, UX was design-led. Increasingly, though, analytics is playing a bigger part. From segmentation to Voice of Customer research to Social Media to Journey Mapping - data provides a scalable view into customer decision-making and choice.
In this huddle, led by Yuriko Medina-Frank of Goldman Sachs, we will consider different techniques for understanding customers and their decision-making, investigate when various methods are most appropriate and share techniques that work well for each method. We will share experiences, good and bad, of using diverse data sets to augment traditional digital analytics data.
The days of pure play digital analytics seem to be coming to an end. Today’s analyst is required to have the skills and experience to handle more than just Adobe or Google Analytics data. As organizations work to integrate their data for a 360o degree view of the customer the need to understand data collection methodologies, data treatment, interpretation and usage becomes increasingly important.
Mia Vallo of National Geographic Partner will be looking to explore the demands on analysts in greater depth. We will be asking:
- What are the essential skills required of a modern-day analyst?
- How does an analyst differ from a data scientist?
- Should analysts look to retrain as data scientists?
- In a world where organizational lines are blurring as more data is integrated, can/should one team manage all disciplines? Or should they be kept as distinct disciplines?
This discussion is intended for both analysts and managers challenged with bridging the gap between analytics disciplines both from a personal and team perspective.
So you have got the best digital tools in the industry, a team stacked with talent, and more actionable insights than you can shake a stick at. But your business partners are still only asking you for data that shows their marketing campaign was the best ever.
In this huddle, led by Rusty Rahmer of Vanguard, we will discuss techniques that are really helping the business think differently and transform into a data driven, customer centric decision-making organization. We will cover methods and best practices that tackle the transformational challenge successfully both from the top down and bottom up.
“Robots cannot take your job if you are already retired.” --Prudential.
That is great if you are over 70 years old and have a pension. But the question for those of us who are still moving up the career ladder, have 29 years left on our mortgage, and are interested in continuing to do digital analytics, is how do we evolve our role to compliment the black box vendor tools with auto-magical ML/AI/insert any other buzz word here.
Thankfully, humans are and always will be needed in the world of digital analytics. Like anything else, over time, we need to evolve to stay relevant. But HOW?
In this huddle, led by Laura Beaulieu of Talbots, we will discuss:
- What will the digital analytics environment look like in a few years?
- What is the value add of “the digital analytics human”?
- What steps should digital analytics professionals consider taking to remain relevant?
- What are the different paths for digital analysts?
- What are the core projects a digital analyst should tackle?
You should walk away with a roadmap for the future – how a digital analytics team can work with the world of automation to ultimately double down on the value add to the company.
Machine learning is evolving rapidly and making great inroads into the commercial, governmental and the voluntary sectors. Organizations are leveraging cloud storage and partnering with vendors in building data capacity that can perform tasks with minimal human intervention.
In this huddle we will examine the role of ML in digital analytics and the use cases helping improve performance:
- What are currently the most common uses of ML within digital analytics?
- What technologies and frameworks are available for analytics teams/marketeers?
- What are the minimal conditions required to build and employ ML in digital analytics?
- How do we integrate ML into our current workflows?
- How is the adoption of these tools and technologies compared to the soaring interest around?
Join us for a fascinating discussion about how ML is changing the life of the data analyst.
Personalization is at an interesting crossroads. Privacy and customer data safeguarding are growing in importance. So how can we use customer information to deliver a personalized and relevant experience without raising privacy concerns?
In this huddle, Rastko Kovacevic from Delta Air Lines will foster a discussion on how to toe the fine line between a surprise-and-delight personalized experience and those experiences that make the customer uncomfortable.
we will look to answer the following questions:
- How to get a personalization program off the ground
- What are the relevant metrics in measuring success
- What tools and technology can be used for personalization
- How to define relevant audiences
- How to hone the messages
- Role of user research in determining the acceptable limits of personalization (focus groups, online forums, etc)
- How to utilize A/B tests in a measured way to optimize personalization experiences This is a must attend huddle for anyone looking to develop a successful personalization program.