alihan/awesome-reviewers
1
0
Fork 0
mirror of https://github.com/baz-scm/awesome-reviewers.git synced 2025-08-20 18:58:52 +03:00
Code Issues Packages Projects Releases Wiki Activity
Files
d30a402e74abe2b2ef045a863cc8d9d250371b17
awesome-reviewers/_reviewers/posthog-ai-context-efficiency.json
internal-baz-ci-app[bot] d30a402e74 Add 41 awesome reviewers for PostHog/posthog
2025-08-19 12:19:58 +00:00

246 lines
23 KiB
JSON
Raw Blame History

[
{
"discussion_id": "2262792462",
"pr_number": 36374,
"pr_file": "products/surveys/backend/prompts.py",
"created_at": "2025-08-08T12:09:10+00:00",
"commented_code": "**Simple NPS**: \"Create an NPS survey\"\n **Targeted Feedback**: \"Get feedback on the dashboard from mobile users\"\n **Complex Research**: \"Survey users about our pricing page experience\"\n+**Feature Flag Targeting**: \"Survey users who have the new-dashboard feature flag enabled\"\n+**Feature Experiment Feedback**: \"Get feedback from beta users in the notebooks-redesign experiment\"",
"repo_full_name": "PostHog/posthog",
"discussion_comments": [
{
"comment_id": "2262792462",
"repo_full_name": "PostHog/posthog",
"pr_number": 36374,
"pr_file": "products/surveys/backend/prompts.py",
"discussion_id": "2262792462",
"commented_code": "@@ -106,6 +112,10 @@\n **Simple NPS**: \"Create an NPS survey\"\n **Targeted Feedback**: \"Get feedback on the dashboard from mobile users\"\n **Complex Research**: \"Survey users about our pricing page experience\"\n+**Feature Flag Targeting**: \"Survey users who have the new-dashboard feature flag enabled\"\n+**Feature Experiment Feedback**: \"Get feedback from beta users in the notebooks-redesign experiment\"",
"comment_created_at": "2025-08-08T12:09:10+00:00",
"comment_author": "marandaneto",
"comment_body": "How will Max AI get the `linked_flag_id` from a `new-dashboard` feature flag key? What else would you need to do to make this work?",
"pr_file_module": null
},
{
"comment_id": "2262833348",
"repo_full_name": "PostHog/posthog",
"pr_number": 36374,
"pr_file": "products/surveys/backend/prompts.py",
"discussion_id": "2262792462",
"commented_code": "@@ -106,6 +112,10 @@\n **Simple NPS**: \"Create an NPS survey\"\n **Targeted Feedback**: \"Get feedback on the dashboard from mobile users\"\n **Complex Research**: \"Survey users about our pricing page experience\"\n+**Feature Flag Targeting**: \"Survey users who have the new-dashboard feature flag enabled\"\n+**Feature Experiment Feedback**: \"Get feedback from beta users in the notebooks-redesign experiment\"",
"comment_created_at": "2025-08-08T12:23:05+00:00",
"comment_author": "marandaneto",
"comment_body": "@PostHog/team-max-ai ping since i am not sure how to proceed here, thanks.",
"pr_file_module": null
},
{
"comment_id": "2262857045",
"repo_full_name": "PostHog/posthog",
"pr_number": 36374,
"pr_file": "products/surveys/backend/prompts.py",
"discussion_id": "2262792462",
"commented_code": "@@ -106,6 +112,10 @@\n **Simple NPS**: \"Create an NPS survey\"\n **Targeted Feedback**: \"Get feedback on the dashboard from mobile users\"\n **Complex Research**: \"Survey users about our pricing page experience\"\n+**Feature Flag Targeting**: \"Survey users who have the new-dashboard feature flag enabled\"\n+**Feature Experiment Feedback**: \"Get feedback from beta users in the notebooks-redesign experiment\"",
"comment_created_at": "2025-08-08T12:34:28+00:00",
"comment_author": "marandaneto",
"comment_body": "Another option would be to allow setting `linked_flag_key` and have our API fetch the id by key, or would Max do that for us?",
"pr_file_module": null
},
{
"comment_id": "2262916666",
"repo_full_name": "PostHog/posthog",
"pr_number": 36374,
"pr_file": "products/surveys/backend/prompts.py",
"discussion_id": "2262792462",
"commented_code": "@@ -106,6 +112,10 @@\n **Simple NPS**: \"Create an NPS survey\"\n **Targeted Feedback**: \"Get feedback on the dashboard from mobile users\"\n **Complex Research**: \"Survey users about our pricing page experience\"\n+**Feature Flag Targeting**: \"Survey users who have the new-dashboard feature flag enabled\"\n+**Feature Experiment Feedback**: \"Get feedback from beta users in the notebooks-redesign experiment\"",
"comment_created_at": "2025-08-08T13:02:25+00:00",
"comment_author": "denakorita",
"comment_body": "Hey @marandaneto, you can use `bind_tools` in order to attach tools to your model and specify what it should use that tool for. In this case your tool would be sth like `retrieve_flag_id(feature_key)` (here the API call to fetch the key)\r\nThe llm basically should be taught to identify the `feature_key` in the user input and then use that to call the tool I mentioned above. \r\nIn order for the llm to identify the `feature_key` you need to maybe provide the entire list of feature keys we have in the prompt (i guess it should not be too long) or have another tool `retrieve_feature_keys` and have the llm do that first then in a second step use the feature key to retrieve its ID. \r\n",
"pr_file_module": null
},
{
"comment_id": "2262960139",
"repo_full_name": "PostHog/posthog",
"pr_number": 36374,
"pr_file": "products/surveys/backend/prompts.py",
"discussion_id": "2262792462",
"commented_code": "@@ -106,6 +112,10 @@\n **Simple NPS**: \"Create an NPS survey\"\n **Targeted Feedback**: \"Get feedback on the dashboard from mobile users\"\n **Complex Research**: \"Survey users about our pricing page experience\"\n+**Feature Flag Targeting**: \"Survey users who have the new-dashboard feature flag enabled\"\n+**Feature Experiment Feedback**: \"Get feedback from beta users in the notebooks-redesign experiment\"",
"comment_created_at": "2025-08-08T13:19:26+00:00",
"comment_author": "marandaneto",
"comment_body": "thanks @denakorita \r\nlemme check if i find some examples about the `bind_tools`",
"pr_file_module": null
},
{
"comment_id": "2262970515",
"repo_full_name": "PostHog/posthog",
"pr_number": 36374,
"pr_file": "products/surveys/backend/prompts.py",
"discussion_id": "2262792462",
"commented_code": "@@ -106,6 +112,10 @@\n **Simple NPS**: \"Create an NPS survey\"\n **Targeted Feedback**: \"Get feedback on the dashboard from mobile users\"\n **Complex Research**: \"Survey users about our pricing page experience\"\n+**Feature Flag Targeting**: \"Survey users who have the new-dashboard feature flag enabled\"\n+**Feature Experiment Feedback**: \"Get feedback from beta users in the notebooks-redesign experiment\"",
"comment_created_at": "2025-08-08T13:23:56+00:00",
"comment_author": "denakorita",
"comment_body": "@marandaneto np, one example that comes to mind is `ee/hogai/graph/taxonomy/nodes.py`, check the prompts there too, yours should be much simpler (that prompt is huge, you maybe dont even need to add tool instructions in the prompt, the tool description should suffice). Let me know if you face any issue, would be happy to help. Ping me via slack. ",
"pr_file_module": null
},
{
"comment_id": "2264933795",
"repo_full_name": "PostHog/posthog",
"pr_number": 36374,
"pr_file": "products/surveys/backend/prompts.py",
"discussion_id": "2262792462",
"commented_code": "@@ -106,6 +112,10 @@\n **Simple NPS**: \"Create an NPS survey\"\n **Targeted Feedback**: \"Get feedback on the dashboard from mobile users\"\n **Complex Research**: \"Survey users about our pricing page experience\"\n+**Feature Flag Targeting**: \"Survey users who have the new-dashboard feature flag enabled\"\n+**Feature Experiment Feedback**: \"Get feedback from beta users in the notebooks-redesign experiment\"",
"comment_created_at": "2025-08-09T19:17:30+00:00",
"comment_author": "lucasheriques",
"comment_body": "@denakorita @marandaneto I took a stab at this problem, and though that, instead of using a LLM call, we could using string matching instead to make it faster and reduce cost. [Here's the commit](https://github.com/PostHog/posthog/pull/36420/commits/45c4aff586d7642c475e85bae0c241521a066252), what do you think?\r\n\r\nAnother alternative is probably get all feature flags on request, and create a dictionary with `ff_key: ff_id` and add that on the prompt.\r\n\r\nI tried adding a tool to fetch that, but at least for me I couldn't make it work reliably",
"pr_file_module": null
},
{
"comment_id": "2266098263",
"repo_full_name": "PostHog/posthog",
"pr_number": 36374,
"pr_file": "products/surveys/backend/prompts.py",
"discussion_id": "2262792462",
"commented_code": "@@ -106,6 +112,10 @@\n **Simple NPS**: \"Create an NPS survey\"\n **Targeted Feedback**: \"Get feedback on the dashboard from mobile users\"\n **Complex Research**: \"Survey users about our pricing page experience\"\n+**Feature Flag Targeting**: \"Survey users who have the new-dashboard feature flag enabled\"\n+**Feature Experiment Feedback**: \"Get feedback from beta users in the notebooks-redesign experiment\"",
"comment_created_at": "2025-08-11T09:09:49+00:00",
"comment_author": "marandaneto",
"comment_body": "> We went looking everywhere, but couldn\u2019t find those commits.\r\n\r\ninvalid commit link",
"pr_file_module": null
},
{
"comment_id": "2266101279",
"repo_full_name": "PostHog/posthog",
"pr_number": 36374,
"pr_file": "products/surveys/backend/prompts.py",
"discussion_id": "2262792462",
"commented_code": "@@ -106,6 +112,10 @@\n **Simple NPS**: \"Create an NPS survey\"\n **Targeted Feedback**: \"Get feedback on the dashboard from mobile users\"\n **Complex Research**: \"Survey users about our pricing page experience\"\n+**Feature Flag Targeting**: \"Survey users who have the new-dashboard feature flag enabled\"\n+**Feature Experiment Feedback**: \"Get feedback from beta users in the notebooks-redesign experiment\"",
"comment_created_at": "2025-08-11T09:11:12+00:00",
"comment_author": "marandaneto",
"comment_body": "> instead to make it faster and reduce cost\r\n\r\nnot sure about the changes if they make sense or not (since commit link is broken), but i'd get it working first and then optimize for the cost if thats ever an issue, assuming the LLM call is the way how all the other Max AI tools do",
"pr_file_module": null
},
{
"comment_id": "2266112175",
"repo_full_name": "PostHog/posthog",
"pr_number": 36374,
"pr_file": "products/surveys/backend/prompts.py",
"discussion_id": "2262792462",
"commented_code": "@@ -106,6 +112,10 @@\n **Simple NPS**: \"Create an NPS survey\"\n **Targeted Feedback**: \"Get feedback on the dashboard from mobile users\"\n **Complex Research**: \"Survey users about our pricing page experience\"\n+**Feature Flag Targeting**: \"Survey users who have the new-dashboard feature flag enabled\"\n+**Feature Experiment Feedback**: \"Get feedback from beta users in the notebooks-redesign experiment\"",
"comment_created_at": "2025-08-11T09:16:15+00:00",
"comment_author": "denakorita",
"comment_body": "@lucasheriques hey lucas, if the list of the feature flags is not very long for the organisations then I would recommend that you build the mapping and inject it to the prompt like u mentioned and avoid tool calling. ",
"pr_file_module": null
},
{
"comment_id": "2266122057",
"repo_full_name": "PostHog/posthog",
"pr_number": 36374,
"pr_file": "products/surveys/backend/prompts.py",
"discussion_id": "2262792462",
"commented_code": "@@ -106,6 +112,10 @@\n **Simple NPS**: \"Create an NPS survey\"\n **Targeted Feedback**: \"Get feedback on the dashboard from mobile users\"\n **Complex Research**: \"Survey users about our pricing page experience\"\n+**Feature Flag Targeting**: \"Survey users who have the new-dashboard feature flag enabled\"\n+**Feature Experiment Feedback**: \"Get feedback from beta users in the notebooks-redesign experiment\"",
"comment_created_at": "2025-08-11T09:20:27+00:00",
"comment_author": "marandaneto",
"comment_body": "> if the list of feature flags is not very long for the organisations\r\n\r\nWe are trading a possible LLM call vs a PostHog API call (always).\r\nThe LLM will have to process the response of the API call even if it is not needed (the prompt does not mention flags but we did give a list of flags).\r\nWe never know how many flags the org will have, can be N or Y.\r\n\r\nI don't know if it pays off.",
"pr_file_module": null
},
{
"comment_id": "2267117312",
"repo_full_name": "PostHog/posthog",
"pr_number": 36374,
"pr_file": "products/surveys/backend/prompts.py",
"discussion_id": "2262792462",
"commented_code": "@@ -106,6 +112,10 @@\n **Simple NPS**: \"Create an NPS survey\"\n **Targeted Feedback**: \"Get feedback on the dashboard from mobile users\"\n **Complex Research**: \"Survey users about our pricing page experience\"\n+**Feature Flag Targeting**: \"Survey users who have the new-dashboard feature flag enabled\"\n+**Feature Experiment Feedback**: \"Get feedback from beta users in the notebooks-redesign experiment\"",
"comment_created_at": "2025-08-11T15:22:26+00:00",
"comment_author": "lucasheriques",
"comment_body": "here's the proper commit, sorry, [wrong link](https://github.com/PostHog/posthog/pull/36420/commits/dd0741118e1d1e7dd791b94a8d5214e2c37bfc36)",
"pr_file_module": null
}
]
},
{
"discussion_id": "2245822928",
"pr_number": 35726,
"pr_file": "ee/hogai/graph/root/prompts.py",
"created_at": "2025-07-31T16:13:04+00:00",
"commented_code": "The tool `create_and_query_insight` generates an arbitrary new query (aka insight) based on the provided parameters, executes the query, and returns the formatted results.\n The tool only retrieves a single query per call. If the user asks for multiple insights, you need to decompose a query into multiple subqueries and call the tool for each subquery.\n \n+CRITICAL ROUTING LOGIC:\n+- On the FIRST request for insights: Perform a search for existing insights first (using `search_insights` tool), then decide whether to use existing ones or create new ones. (remember to set state.root_tool_insight_plan to the user's query)",
"repo_full_name": "PostHog/posthog",
"discussion_comments": [
{
"comment_id": "2245822928",
"repo_full_name": "PostHog/posthog",
"pr_number": 35726,
"pr_file": "ee/hogai/graph/root/prompts.py",
"discussion_id": "2245822928",
"commented_code": "@@ -58,6 +58,11 @@\n The tool `create_and_query_insight` generates an arbitrary new query (aka insight) based on the provided parameters, executes the query, and returns the formatted results.\n The tool only retrieves a single query per call. If the user asks for multiple insights, you need to decompose a query into multiple subqueries and call the tool for each subquery.\n \n+CRITICAL ROUTING LOGIC:\n+- On the FIRST request for insights: Perform a search for existing insights first (using `search_insights` tool), then decide whether to use existing ones or create new ones. (remember to set state.root_tool_insight_plan to the user's query)",
"comment_created_at": "2025-07-31T16:13:04+00:00",
"comment_author": "kappa90",
"comment_body": "the LLM doesn't know what `state.root_tool_insight_plan` is, it can only do tool calls specifying an arg (`query_description` in the case of `create_and_query_insight`)",
"pr_file_module": null
},
{
"comment_id": "2246003978",
"repo_full_name": "PostHog/posthog",
"pr_number": 35726,
"pr_file": "ee/hogai/graph/root/prompts.py",
"discussion_id": "2245822928",
"commented_code": "@@ -58,6 +58,11 @@\n The tool `create_and_query_insight` generates an arbitrary new query (aka insight) based on the provided parameters, executes the query, and returns the formatted results.\n The tool only retrieves a single query per call. If the user asks for multiple insights, you need to decompose a query into multiple subqueries and call the tool for each subquery.\n \n+CRITICAL ROUTING LOGIC:\n+- On the FIRST request for insights: Perform a search for existing insights first (using `search_insights` tool), then decide whether to use existing ones or create new ones. (remember to set state.root_tool_insight_plan to the user's query)",
"comment_created_at": "2025-07-31T17:37:53+00:00",
"comment_author": "tatoalo",
"comment_body": "Yes @kappa90, totally right here as well! This was just a test that I performed but forgot to remove, it indeed had no effect at all :) ",
"pr_file_module": null
}
]
},
{
"discussion_id": "2251887833",
"pr_number": 35726,
"pr_file": "ee/hogai/graph/insights/nodes.py",
"created_at": "2025-08-04T15:51:01+00:00",
"commented_code": "root_tool_call_id=None,\n )\n \n- def router(self, state: AssistantState) -> Literal[\"end\", \"root\"]:\n+ def _evaluate_insights_for_creation(self, selected_insights: list[int], user_query: str) -> dict:",
"repo_full_name": "PostHog/posthog",
"discussion_comments": [
{
"comment_id": "2251887833",
"repo_full_name": "PostHog/posthog",
"pr_number": 35726,
"pr_file": "ee/hogai/graph/insights/nodes.py",
"discussion_id": "2251887833",
"commented_code": "@@ -313,7 +528,149 @@ def _create_error_response(self, content: str, tool_call_id: str | None) -> Part\n root_tool_call_id=None,\n )\n \n- def router(self, state: AssistantState) -> Literal[\"end\", \"root\"]:\n+ def _evaluate_insights_for_creation(self, selected_insights: list[int], user_query: str) -> dict:",
"comment_created_at": "2025-08-04T15:51:01+00:00",
"comment_author": "kappa90",
"comment_body": "nit: this tells you if the set of insights is useful, rather than telling which specific ones should be kept; if you moved to tool calls, you could easily ask the LLM to select the right ones directly, with explanation and everything.",
"pr_file_module": null
},
{
"comment_id": "2256958546",
"repo_full_name": "PostHog/posthog",
"pr_number": 35726,
"pr_file": "ee/hogai/graph/insights/nodes.py",
"discussion_id": "2251887833",
"commented_code": "@@ -313,7 +528,149 @@ def _create_error_response(self, content: str, tool_call_id: str | None) -> Part\n root_tool_call_id=None,\n )\n \n- def router(self, state: AssistantState) -> Literal[\"end\", \"root\"]:\n+ def _evaluate_insights_for_creation(self, selected_insights: list[int], user_query: str) -> dict:",
"comment_created_at": "2025-08-06T12:14:48+00:00",
"comment_author": "tatoalo",
"comment_body": "Great point @kappa90! Refactoring to using more tool calls!",
"pr_file_module": null
}
]
},
{
"discussion_id": "2267375367",
"pr_number": 36420,
"pr_file": "products/surveys/backend/max_tools.py",
"created_at": "2025-08-11T16:32:45+00:00",
"commented_code": "async def _create_survey_from_instructions(self, instructions: str) -> SurveyCreationSchema:\n \"\"\"\n- Create a survey from natural language instructions.\n+ Create a survey from natural language instructions using PostHog-native pattern.\n \"\"\"\n- # Create the prompt\n- prompt = ChatPromptTemplate.from_messages(\n- [\n- (\"system\", SURVEY_CREATION_SYSTEM_PROMPT),\n- (\"human\", \"Create a survey based on these instructions: {{{instructions}}}\"),\n- ],\n- template_format=\"mustache\",\n- )\n+ logger.info(f\"Starting survey creation with instructions: '{instructions}'\")\n+\n+ # Extract and lookup feature flags inline (following PostHog pattern)\n+ feature_flag_context = await self._extract_feature_flags_inline(instructions)\n+\n+ # Build enhanced system prompt with feature flag information\n+ enhanced_system_prompt = SURVEY_CREATION_SYSTEM_PROMPT\n+ if feature_flag_context:\n+ enhanced_system_prompt += f\"\n\n## Available Feature Flags\n{feature_flag_context}\"\n+\n+ # Single LLM call with all context (cost-effective, fast)\n+ prompt = ChatPromptTemplate.from_messages([\n+ (\"system\", enhanced_system_prompt),\n+ (\"human\", \"Create a survey based on these instructions: {{{instructions}}}\")\n+ ], template_format=\"mustache\")\n \n- # Set up the LLM with structured output\n model = (\n- ChatOpenAI(model=\"gpt-4.1-mini\", temperature=0.2)\n+ ChatOpenAI(model=\"gpt-5-mini\")",
"repo_full_name": "PostHog/posthog",
"discussion_comments": [
{
"comment_id": "2267375367",
"repo_full_name": "PostHog/posthog",
"pr_number": 36420,
"pr_file": "products/surveys/backend/max_tools.py",
"discussion_id": "2267375367",
"commented_code": "@@ -42,34 +45,38 @@ class CreateSurveyTool(MaxTool):\n \n async def _create_survey_from_instructions(self, instructions: str) -> SurveyCreationSchema:\n \"\"\"\n- Create a survey from natural language instructions.\n+ Create a survey from natural language instructions using PostHog-native pattern.\n \"\"\"\n- # Create the prompt\n- prompt = ChatPromptTemplate.from_messages(\n- [\n- (\"system\", SURVEY_CREATION_SYSTEM_PROMPT),\n- (\"human\", \"Create a survey based on these instructions: {{{instructions}}}\"),\n- ],\n- template_format=\"mustache\",\n- )\n+ logger.info(f\"Starting survey creation with instructions: '{instructions}'\")\n+\n+ # Extract and lookup feature flags inline (following PostHog pattern)\n+ feature_flag_context = await self._extract_feature_flags_inline(instructions)\n+\n+ # Build enhanced system prompt with feature flag information\n+ enhanced_system_prompt = SURVEY_CREATION_SYSTEM_PROMPT\n+ if feature_flag_context:\n+ enhanced_system_prompt += f\"\\n\\n## Available Feature Flags\\n{feature_flag_context}\"\n+\n+ # Single LLM call with all context (cost-effective, fast)\n+ prompt = ChatPromptTemplate.from_messages([\n+ (\"system\", enhanced_system_prompt),\n+ (\"human\", \"Create a survey based on these instructions: {{{instructions}}}\")\n+ ], template_format=\"mustache\")\n \n- # Set up the LLM with structured output\n model = (\n- ChatOpenAI(model=\"gpt-4.1-mini\", temperature=0.2)\n+ ChatOpenAI(model=\"gpt-5-mini\")",
"comment_created_at": "2025-08-11T16:32:45+00:00",
"comment_author": "denakorita",
"comment_body": "How was the latency here? Also, did u have to upgrade `langchain` to use gpt5? \r\nMaybe also setting the `reasoning={\"effort\": \"low\"},` here might be best. ",
"pr_file_module": null
}
]
}
]
Reference in New Issue View Git Blame Copy Permalink