Help
Options
Didn't know it?
click below
click below
Knew it?
click below
click below
Don't Know
Remaining cards (0)
Know
retry
shuffle
restart
0:00
Embed Code - If you would like this activity on your web page, copy the script below and paste it into your web page.
Normal Size Small Size show me how
Normal Size Small Size show me how
Product Analyst
Interview Questions for Product Analyst
| Question | Answer |
|---|---|
| What does the pvalue measure in a regression analysis | Pvalue measures the probability that your results are driven by random chance. The lower the pvalue the better |
| How would you determine seasonality for a product | Review a KPI that would change across quarters like sales or total revenue generated. If the increased revenue is consistent across years within the same period, the product likely has seasonality.. Test a seasonal variable and observe change in Y |
| In a nutshell, what do you do at Oracle | Extract meaningful insights for weekly/monthly reporting and perform more detailed analysis for product & marketing teams to drive business decisions. Create and deliver custom dashboards to product marketing teams and also enable teams on how to use |
| Can you tell us more about pillar wide system QA you perform and how it leads to reduced variances in reporting? | Perform a bi-weekly sanity check on all my reporting metrics from all my dashboards to identify any significant spikes or drops and investigate the reason. Also be ensure that all dashboards are up to date with latest data changes and methodologies |
| What kind of assets do you put together for the QBR | Mainly around events performance (registrations, attendees, ratios, top performing) for each region and a YoY/YTD analysis. Also report on pipeline in the same way, highlighting top priority open accounts to target for next quarter |
| How do you manage the budget? What are your key tasks? | Track quarterly budget and what has been forecasted vs actually spent. I join budget data with pipeline data in order to provide robust ROI analysis across product pillars & programs to identify top generating programs/pillars to advise on future spend |
| What do you do for customer insights? How do you use these insights to drive business decisions? | Two main parts: win loss analysis work done on ADB war room and utilizing customer behavior from customer responses to identify possible opportunities. Obtain roles, industries, title customer data to advise productmarketing teams on key target audiences |
| Can you give more details around your role as the ADB War room Lead | First stab at providing OCI product performance on a global level to drive business decisions. Managed team of five to report on trails, pipe, consumption & win loss analysis. Used W/L analysis to identify trends in competitors and product combinations |
| Can you elaborate on your dashboarding process? What are your key roles? | Create, maintain, and own enablement around several key dashboards used globally by the product marketing org. Use SQL to create custom calculations to arrive at visualizations tailored specifically for each team. Created to reduce time spent on reporting |
| What is your experience with AA? How do you increase web traffic? | Work with digital design team to ensure that all significant marketing assets are promoted properly on all channels. Identify low performing assets to identify channels where it has not yet been promoted. Move assets around on o.com based on traffic trend |
| What does the coefficient measure? How is it different from Rsq? | Based on the data input, coefficient measures the impact of y given one unit x. R-squared is a statistical measure that represents the proportion of the variance for a dependent variable that's explained by an independent variable or variables in a regres |
| How is the Tstat calculated? What does the Tstat measure? | Measured by dividing the coefficient by std error. The higher the tstat t(abs value) the more likely you should reject the null hypothesis and that there is statistical significance in your data. It is the departure of est value from zero |
| What does the ANOVA section mean? What are key metrics? | ANOVA = an analysis of a variance. ANOVA section is good area to obtain high level understanding of the model (look at Model). Key metrics are SS, df, and mean square. Rsq =SS Model/SS Total |
| What is the adjusted Rsq (ARSQ)? How is it different from Rsq and when should you use it | Adjusted Rsq is different from Rsq that it takes into account adding additional x variables. Rsq only increases with added variables so the str of a model is misrepresented. ARSQ generally decreases with added variables |
| What is the null hypothesis | null hypothesis states that there is no relationship between two variables being studied. Doing linear regressions you are generally trying to disprove the null hypothesis. Higher pvalue = more likely to ACCEPT null hypothesis |
| What is SSR, SSE, SST and how do you calculate Rsq | SSR = Sum of all sq. that can be exp. by model SSE = sum of all sq. that cant be exp. by model SST = Sum of all sq. for total diff Rsq = SSR/SST - explained differences/total differences |
| What is a regression Analysis? Have you performed one? | Regression analysis is normally performed on top of an AB test to identify the statistical significance between your variables. I do not have real world exp. in using one but understand the concepts and how to read it. Have played with f(x) in excel & OAC |
| Why are you a good fit for this role even though you dont have experience in AB testing or regression analysis | My other responsibilities and experiences at oracle are aligned to this role. Report & Collab with same teams, utilize data to provide insights, dashboarding. I understand conceptually what an AB test is and how to read a regression analysis |
| What is your experience in SQL? How do you use it at your company? | All our databases and analytics tools utilize SQL. I work mainly with low code dev. app so I do not need to write code manually but am able to comfortably read syntax used. Use SQL to create many custom equations and data queries |
| What is a power analysis and what does it help identify? What are the four components of a power analysis? | Power analysis measures the statistical power your model has - it helps provide a probability on how accurate your pvalue is. It is used to determine sample size. Four components are: effect size, significance %, power lvl, sample size |
| What is the order of power analysis, regression analysis and AB testing you would perform for the best results & why | Power analysis first to determine sample size needed for accuracy of results, perform AB test using sample size calculated in power. Then a regression analysis to obtain insights on how much of the results can be attributed to change in x variable |
| Walk us through what you do to perform research initiatives to drive business strategy | Lots of our higher lvl analysis is automated by our custom dashboards. We do deeper level analysis when prescribed by exec. management for a business question. Very collaborative and responsibilities evolve over time |
| Can you tell us a little about your product mapping project? What did it accomplish in the end? | Collaborated with sales and product teams to ensure that the 300+ products under the pillar rollups between the sales DB and CXD DB align for accurate pipeline reporting. performed reconciliation between mappings & pipe once completed. |
| How do you use customer data to drive business insights. Can you give us one example>? | By combining product information with customer information, I'm able to extract which customer personas are consuming the most content for a certain product |
| What are some insights you use Adobe Analytics for? | Identify videos/assets that are underperforming and follow up with marketing teams to ensure that the video/asset gets promoted via email campaigns, paid media, or even repositioned on o.com |
| Why are you interested in Bolt? In your words, what does Bolt do? | Like how it is within the tech industry but has an ecommerce aspect to it. Strongly believe in its success as the ecommerce industry just keeps on growing. Passionate to work at a company where the product is something I can relate to and grow |
| Why are you a good fit for this role? | Much of my experience at Oracle overlaps with the responsibilities of this position. Same teams to collab with, data driven, focused around KPIs and improving business efficiency and success. Logical thinker that strives to make an actual impact |
| Can you talk about work you've done in regards to the pipeline attribution model, data defining and data suppression rules? | - Collaborated with sales and BI team to create response matching system between opportunity and marketing program - Built the infrastructure for proper classification of key marketing fields used in data analysis. Same for data suppression |
| Walk us through your typical day at Oracle | - Assist product & marketing teams on any of their weekly reporting needs - Maintain dashboards with latest BI changes. Ensure that additional visualizations/calculations are created - enablement Q&A |
| What is an example of an AB test you would run for Bolt? What KPI would you measure and what variables would you test? How would you interpret the results? | Test whether changing the Apt # field to not having to be clicked on to appear would increase conversions or decrease total time to checkout for first time users. If time to checkout decreases or conversion rate inc. we change it to non-click-to-open |
| What are some key reasons an AB test would fail? What does a successful AB test look like? | Fail: - not well defined metrics - running the test for too short a period - testing variables that do not relate to selected metric - incorrectly interpret the results - incorrect hypothesis Success: - the opposite |
| Why are AB tests important for businesses? | Assist businesses in making business decisions to arrive closer to optimal performance through comparative analysis of two+ solutions. Provides real data on how specific levers within the company impact key business metrics |
| How long should you run your AB test for? | Generally the longer, the more accurate your results will be. At the bare minimum, your test should run for as long as it takes to obtain required number of obs. to attain desired statistical power which is determined at planning |
| How do you work with big data at your company? How do you manage large amounts of data to provide accurate and meaningful insights | - Utilize dashboards with many SQL driven custom formulas to handle majority of high level reporting - use data flows to power these dashboards to reduce time spent refreshing queries - join multiple data sets to perform x-analysis |
| How do you interpret regression analysis - what are some key metrics to understand | pvalue - states how stat. significant your model is. probability that results happened purely by chance Rsq - shows how much of var. can be exp. by your model Statistical power - % of confidence in accurate pvalue Tstat - evidence against null hypot |
| What are the differences between an inner join, left join, right join, and all join? | - Inner J: pulls results only for if matches are found in both database - Left/Right J: basically the same depending on how you write syntax. Pulls all results for left DB but only matching results for right - Pulls all results from both DB |
| How would you put together a data analysis how the number of fields on checkout impact total # of sales? | Run an AB test on how # of fields (variable) impact total # of sales (KPI). To test the stat. significance of the results obtained, you can run a regression analysis on the data observed |
| What significance level would you target in an AB test | Generally .05 but can be increased or decreased based on the tester. If result will drive a big business decision costing lots of time/money may want to increase significance level to .01. Can also raise the required amt of stat. power |
| Lets say that your company is running a standard control and variant AB test on a feature to increase conversion rates on the landing page. PM checks the result and finds a .04P value. How would you assess the validity of the result | If not too costly, see if you can replicate the results. You can also run a power analysis to see the statistical power your regression model has to be more confident in the correctly calc. p value |
| What is the primacy effect? What is the novelty effect? How can these situations be avoided? | - Primacy Eff: lower then exp. outcome caused by natural resis. to change - Novelty Eff: higher then exp. outcome by initial excitement/novelty - Avoid by running test longer |
| What are ways you can test whether or not a product has seasonality? | Observe total conversions/sales/revenue across a year to identify whether certain quarters has inc. or decrease. See if it can be replicated. AB test on sales using variable with seasonality component (temp) |
| Define how you would choose key metrics for an AB test. Please elaborate on the process and explain your reasoning. | - must answer business question and reach bottom line - KPI must be defined enough to not be influenced by ext. variables - variables must impact KPI - difficulty/cost/time in obtaining observations |
| What are some key metrics you would consider performing an AB test on for ecommerce purchases. Explain your reasoning | - conversion rate - CTR - engagement time - time between landing page & checkout - overall visits resulting from digital ads |
| You are changing the layout to the webpage. All things equal and same, which KPIs would it make sense to monitor and why? Give two examples. | # of clicks when on the website and total web page engagement time |
| How would you approach designing an AB test and what factors would you consider | - Defining the KPI - Defining the variables - Create hypothesis - sample size & duration of testing - difficulty of running test - randomization & assignment |
| What is the combined total uplift of all your tests? How should you measure this? | If the net effect of your combined individual experiments is positive - with stat. significance. It wouldnt be necessarily a bad idea to implement all of them. If you wanted to double check, you can run a multivariant (combined) test |
| So what’s the actual business impact over the last 12 months? What do you do if after implementation the real world results do not occur in the way you want them? | Important to consider once the change is made and not done in a controlled environment, there can be many more factors impacting the result. Not all tests will have the same result in the real world practice. Just keep moving on |
| What are possible reasons we might not see a lift that we observed in AB testing | Statistical significance does not necessarily mean validity. Even if we observe a lift on a SAMPLE pop. it does not guarantee we will see that same lift on the entire pop. Be wary of false positives as well which can be reduced in probab. by power analys |
| When using aggregate functions what is something you absolutely have to use it with | GROUP BY function or else the aggregate function will come out as just one value |
| Tell us about yourself. What are your strengths, what are your weaknesses? | detail oriented, critical thinking, hard worker. Very passionate about the work I perform and exceeding expectations for the team. Weaknesses can be tunnel focus - hard to pull myself out of one task but PM roles have improved me |
| Can you walk us through an analysis you performed at your company that benefited the organization? | Lead Analysis example |
| What is your greatest strength as a product analyst? | A lot of experience working with product & marketing teams around product performance across multiple KPIs. Have defined, tracked and reported on the full customer funnel from responses to pipeline including web traffic. Data driven with technical skills |
| Talk about a time you set a goal and you were able to meet or achieve it? | Either the CPA or the global tech pillar dashboard - ask first if it is personal |
| Why are you interested in working as a product analyst here at Bolt? | Want to gain product focus and experience. At oracle we work with so many different products with varying complexities and its like being jack of all trades but master of none. Want to provide real business insight to see tangible results of a product |
| What motivates you at the work place | Good manager and team, belief in the product, feeling my work is fulfilling and I'm being properly appreciated |
| Why are you thinking of leaving Oracle | Not really thinking about it until this job came along. I've always wanted to work at a startup where I can really drive the direction of the company up front. Not applying to other jobs and solely have a one off interest in BOLT |
| What is your greatest achievement so far? | Greatest achievement would be having the confidence and drive to switch roles from internal audit and BP consulting to product marketing analytics. |
| Tell me about a time you failed | Karen's dashboarding request |
| What are important traits a product analyst should have to be successful | Collaboration is key as you'll be working cross functionally with the product, data, and marketing teams to identify key features/gaps to establish and how best to bring those to market. Detail oriented and critical thinking to develop structured AB tests |
| What skills do you think you bring to the table | detail oriented and data driven, skilled at time & project management, fast/efficient worker, can do attitude and a team player |
| What do you do at your company to reduce costs, increase revenue or save time | ROI analysis to see where money is being spent without generating a substantial return, create dashboards to provide high level reporting across multiple LOBs and product pillars to reduce their time spent, pipeline analysis |
| What would you describe as the core features of ecommerce? What things do companies need to get right in order to be successful | - customer UI - digital marketing - security - bridging the gap between commerce and ecommerce in terms of reliability of the product being delivered |
| What are some of the main differences between ecommerce and traditional commerce | ecommerce is more accessible, you dont need to go to a store to get what you want but also the item doesnt arrive the moment you pay for it. ecommerce requires the user to be a bit more technical and understand how to navigate the businesses' UI |
| What are some key metrics to track in any ecommerce model | - conversion rates - # of sales - # of subscribers/returning customers - CTRs |
| What are some disadvantages ecommerce and how would you try and resolve them | Buying in store seems very simple and everyone can do it. Best way to bridge this gap is fixing the UI of your ecommerce business and make as simple and easy as possible while providing adv. specific to ecommerce as well |
| We notice that customer churn is on the rise, how would you boost repeat purchase/renewal rates and increase customer retention | Run customer satisfaction surveys. It means a lot to a customer when a business is proactive in obtaining customer satisfaction once theyve already been won. Also it provides valuable insights in our product areas we can improve on. Analyze competition |
| How do you ensure to get your message over to programmers, marketers, and other staff members from the product development team? | Do some data analysis to see who are the main consumers of the product assets that are available online. This can be done by aggregating customer data for gated assets. Tailor messaging based on customer persona (title, industry, role) |
| Describe your process of identifying KPI (key performance indicators) for the product. | Know the bottom line of the business question youre trying to solve. Identify variables that you think would directly impact KPI chosen. KPI must not be too broad to avoid being impacted by external variables. Create a hypothesis for results |
| What do you know about the products of our company? How would you describe them? What are some ways you would change it? | Platform solution for online businesses to enhance the customer experience in regards to the checkout process. Reduce dropped carts by increasing conversions by streamlining checkout procedures |
| How would you measure the success of a product launch? | It depends on what aspect of a product launch. Are we testing the amount of reach/attention/traffic or the long term events on sales/product If the latter, observe success after launch excitement wears down to avoid primacy/novelty effects |
| How do you imagine a typical day in this job? | Would be pulling, organizing and interpreting product performance on key KPIs and communicating results & findings to product teams. Working with product teams to create AB tests to run and performing analysis on results |
| In your opinion, who is our main competitor, and what do they do better than us? | paypal, fast, stripe I believe to be the main competitors. Main advantage paypal has on bolt is that its not a tailored and easier to implement. Much more universally known with legacy too. Stripe is more dev. focused |
| Tell us about a time when you showed initiative at work. | talent rotational program at work |
| How many cabs are on the streets of London at this moment? (brain teaser question) | # of cabs/people in the US. Find how the total % of cabs were impacted after uber. Multiple cabs/people by population in London and adjust for the rideshare impact on london. Could also do it by square feet |
| What are AB tests and have you used them in your work? | AB test is a way to test different models/versions of a product or webpage UI to see what changes can be made to product or feature to increase product success. Not a formal AB test but I have activities similar to the AB test that I perform |
| What do you think the future of ecommerce would be like? What changes should businesses be ready to adapt for? | Ecommerce only continues to grow as people move away from physical - esp with the recent year and covid 19. Ecommerce will slowly evolve to bridge the gap between trad. commerce w. augmented reality and real time service. security |
| Name strengths of Paypal, fast, Stripe and what we can do to compete with them | |
| What would you do to increase the # of customers that enable Bolt Account creations | If we give the customers an excellent experience, this will increase our value to businesses. Creating content around customer exp. with Bolt to educate them on how it works and how it is safe that their data is being shared across bolt network |
| What would you change about our product in regards to Bolt Account creation? | Create more buzz about bolt and its network in the consumer realm and allow customers to create bolt accounts even prior to checking out at a partnered business. The larger our bolt network the more appealing it would be to businesses |
Created by:
turtle129