mirror of
https://github.com/microsoft/LLMLingua.git
synced 2024-01-23 02:05:46 +03:00
413 lines
27 KiB
Plaintext
413 lines
27 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "1972a352-a0e3-41b7-81dc-dd4ae2b890c3",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Code Completion"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "05d999bc-83a3-454f-a8a4-44cbff1fcedc",
|
|
"metadata": {},
|
|
"source": [
|
|
"<a target=\"_blank\" href=\"https://colab.research.google.com/github/microsoft/LLMLingua/blob/main/examples/Code.ipynb\">\r\n",
|
|
" <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/>\r\n",
|
|
"</a>"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "fe3ed1ce-d38d-4048-9db6-9707b55dc642",
|
|
"metadata": {},
|
|
"source": [
|
|
"In addition to natural language, we also evaluate the capabilities of our method on code. For example, in [**repobench-p**](https://arxiv.org/abs/2306.03091), the original prompts can be particularly long, up to **20k** characters. LongLLMLingua can achieve a 1.4 point improvement at a **6x** compression ratio."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "18422597-687a-43aa-a6ed-ce6244d0eb55",
|
|
"metadata": {},
|
|
"source": [
|
|
"### RepoBench-p"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "51a7accd-5ec2-4ed2-9582-1afdb441a998",
|
|
"metadata": {},
|
|
"source": [
|
|
"Next, we will demonstrate the use of LongLLMLingua on the **RepoBench-P** dataset (Code Completion Task). This dataset contains samples of incomplete code snippets and several related reference code files. The task is to supplement and complete the next line of code. \n",
|
|
" \n",
|
|
"The original dataset can be found at https://huggingface.co/datasets/THUDM/LongBench."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"id": "a970a901-11bd-43af-a8bc-7fb2fc6a1a07",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Defaulting to user installation because normal site-packages is not writeable\n",
|
|
"Requirement already satisfied: llmlingua in /home/hjiang/Code/github/LLMLingua (0.1.2)\n",
|
|
"Requirement already satisfied: datasets in /home/hjiang/.local/lib/python3.9/site-packages (2.14.4)\n",
|
|
"Requirement already satisfied: nltk in /home/hjiang/.local/lib/python3.9/site-packages (from llmlingua) (3.8.1)\n",
|
|
"Requirement already satisfied: numpy in /home/hjiang/.local/lib/python3.9/site-packages (from llmlingua) (1.23.5)\n",
|
|
"Requirement already satisfied: tiktoken in /home/hjiang/.local/lib/python3.9/site-packages (from llmlingua) (0.4.0)\n",
|
|
"Requirement already satisfied: torch in /home/hjiang/.local/lib/python3.9/site-packages (from llmlingua) (1.13.1+cu116)\n",
|
|
"Requirement already satisfied: transformers>=4.26.0 in /home/hjiang/.local/lib/python3.9/site-packages (from llmlingua) (4.34.1)\n",
|
|
"Requirement already satisfied: pyarrow>=8.0.0 in /home/hjiang/.local/lib/python3.9/site-packages (from datasets) (11.0.0)\n",
|
|
"Requirement already satisfied: dill<0.3.8,>=0.3.0 in /home/hjiang/.local/lib/python3.9/site-packages (from datasets) (0.3.7)\n",
|
|
"Requirement already satisfied: pandas in /home/hjiang/.local/lib/python3.9/site-packages (from datasets) (2.0.3)\n",
|
|
"Requirement already satisfied: requests>=2.19.0 in /home/hjiang/.local/lib/python3.9/site-packages (from datasets) (2.29.0)\n",
|
|
"Requirement already satisfied: tqdm>=4.62.1 in /home/hjiang/.local/lib/python3.9/site-packages (from datasets) (4.65.0)\n",
|
|
"Requirement already satisfied: xxhash in /home/hjiang/.local/lib/python3.9/site-packages (from datasets) (3.3.0)\n",
|
|
"Requirement already satisfied: multiprocess in /home/hjiang/.local/lib/python3.9/site-packages (from datasets) (0.70.15)\n",
|
|
"Requirement already satisfied: fsspec[http]>=2021.11.1 in /home/hjiang/.local/lib/python3.9/site-packages (from datasets) (2023.6.0)\n",
|
|
"Requirement already satisfied: aiohttp in /home/hjiang/.local/lib/python3.9/site-packages (from datasets) (3.8.5)\n",
|
|
"Requirement already satisfied: huggingface-hub<1.0.0,>=0.14.0 in /home/hjiang/.local/lib/python3.9/site-packages (from datasets) (0.16.4)\n",
|
|
"Requirement already satisfied: packaging in /home/hjiang/.local/lib/python3.9/site-packages (from datasets) (23.0)\n",
|
|
"Requirement already satisfied: pyyaml>=5.1 in /usr/lib/python3/dist-packages (from datasets) (5.3.1)\n",
|
|
"Requirement already satisfied: attrs>=17.3.0 in /home/hjiang/.local/lib/python3.9/site-packages (from aiohttp->datasets) (23.1.0)\n",
|
|
"Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /home/hjiang/.local/lib/python3.9/site-packages (from aiohttp->datasets) (3.2.0)\n",
|
|
"Requirement already satisfied: multidict<7.0,>=4.5 in /home/hjiang/.local/lib/python3.9/site-packages (from aiohttp->datasets) (6.0.4)\n",
|
|
"Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /home/hjiang/.local/lib/python3.9/site-packages (from aiohttp->datasets) (4.0.2)\n",
|
|
"Requirement already satisfied: yarl<2.0,>=1.0 in /home/hjiang/.local/lib/python3.9/site-packages (from aiohttp->datasets) (1.9.2)\n",
|
|
"Requirement already satisfied: frozenlist>=1.1.1 in /home/hjiang/.local/lib/python3.9/site-packages (from aiohttp->datasets) (1.4.0)\n",
|
|
"Requirement already satisfied: aiosignal>=1.1.2 in /home/hjiang/.local/lib/python3.9/site-packages (from aiohttp->datasets) (1.3.1)\n",
|
|
"Requirement already satisfied: filelock in /home/hjiang/.local/lib/python3.9/site-packages (from huggingface-hub<1.0.0,>=0.14.0->datasets) (3.12.2)\n",
|
|
"Requirement already satisfied: typing-extensions>=3.7.4.3 in /home/hjiang/.local/lib/python3.9/site-packages (from huggingface-hub<1.0.0,>=0.14.0->datasets) (4.7.1)\n",
|
|
"Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3/dist-packages (from requests>=2.19.0->datasets) (2.8)\n",
|
|
"Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/hjiang/.local/lib/python3.9/site-packages (from requests>=2.19.0->datasets) (1.26.16)\n",
|
|
"Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3/dist-packages (from requests>=2.19.0->datasets) (2019.11.28)\n",
|
|
"Requirement already satisfied: regex!=2019.12.17 in /home/hjiang/.local/lib/python3.9/site-packages (from transformers>=4.26.0->llmlingua) (2023.6.3)\n",
|
|
"Requirement already satisfied: tokenizers<0.15,>=0.14 in /home/hjiang/.local/lib/python3.9/site-packages (from transformers>=4.26.0->llmlingua) (0.14.1)\n",
|
|
"Requirement already satisfied: safetensors>=0.3.1 in /home/hjiang/.local/lib/python3.9/site-packages (from transformers>=4.26.0->llmlingua) (0.3.1)\n",
|
|
"Requirement already satisfied: click in /home/hjiang/.local/lib/python3.9/site-packages (from nltk->llmlingua) (8.1.6)\n",
|
|
"Requirement already satisfied: joblib in /home/hjiang/.local/lib/python3.9/site-packages (from nltk->llmlingua) (1.3.1)\n",
|
|
"Requirement already satisfied: python-dateutil>=2.8.2 in /home/hjiang/.local/lib/python3.9/site-packages (from pandas->datasets) (2.8.2)\n",
|
|
"Requirement already satisfied: pytz>=2020.1 in /home/hjiang/.local/lib/python3.9/site-packages (from pandas->datasets) (2023.3)\n",
|
|
"Requirement already satisfied: tzdata>=2022.1 in /home/hjiang/.local/lib/python3.9/site-packages (from pandas->datasets) (2023.3)\n",
|
|
"Requirement already satisfied: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil>=2.8.2->pandas->datasets) (1.14.0)\n",
|
|
"\n",
|
|
"\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.2.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.3.1\u001b[0m\n",
|
|
"\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpython3.9 -m pip install --upgrade pip\u001b[0m\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"# Install dependency.\n",
|
|
"!pip install llmlingua datasets"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"id": "628a0a2a-aa17-4f12-81c5-f09a534a434f",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Download the original prompt and dataset\n",
|
|
"from datasets import load_dataset\n",
|
|
"dataset = load_dataset('THUDM/LongBench', \"repobench-p\", split='test')"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"id": "cbbbf3de-a9d6-46cf-afab-dcb72a6154ec",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Using the OAI\n",
|
|
"import openai\n",
|
|
"openai.api_key = \"<insert_openai_key>\""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 14,
|
|
"id": "46506810-8565-43da-984b-d862c56b49c2",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# or Using the AOAI\n",
|
|
"import openai\n",
|
|
"openai.api_key = \"<insert_openai_key>\"\n",
|
|
"openai.api_base = \"https://xxxx.openai.azure.com/\"\n",
|
|
"openai.api_type = 'azure'\n",
|
|
"openai.api_version = '2023-05-15'"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "f8676ffa-5117-44dc-9742-bb9ab1d56e0c",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Setup Data"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 12,
|
|
"id": "bb349566-83d8-44ac-a683-b67ed9ddf7a6",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# select an example from MeetingBank\n",
|
|
"contexts, question, answer = [dataset[1][key] for key in [\"context\", \"input\", \"answers\"]]\n",
|
|
"instruction = \"Please complete the code given below.\"\n",
|
|
"question = question + \"\\n\\nNext line of code:\\n\""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 13,
|
|
"id": "58718a19-cc4e-4002-a92a-58ea3de9c9d0",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"[' mEffects.add(new NoEffect());']"
|
|
]
|
|
},
|
|
"execution_count": 13,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"# Ground-truth Answer\n",
|
|
"answer"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ba1c6d52-dc87-434c-a41c-0bbc8a286504",
|
|
"metadata": {},
|
|
"source": [
|
|
"### The response of Original prompt"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 16,
|
|
"id": "3d441f10-c5c7-4d45-b09a-717e536b36bf",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"{\n",
|
|
" \"id\": \"chatcmpl-8FjSIsMsihREbjFpWGGSHgT6OpNqK\",\n",
|
|
" \"object\": \"chat.completion\",\n",
|
|
" \"created\": 1698760342,\n",
|
|
" \"model\": \"gpt-4-32k\",\n",
|
|
" \"choices\": [\n",
|
|
" {\n",
|
|
" \"index\": 0,\n",
|
|
" \"finish_reason\": \"stop\",\n",
|
|
" \"message\": {\n",
|
|
" \"role\": \"assistant\",\n",
|
|
" \"content\": \"The next line of code is missing. Please provide more information.\"\n",
|
|
" }\n",
|
|
" }\n",
|
|
" ],\n",
|
|
" \"usage\": {\n",
|
|
" \"prompt_tokens\": 18817,\n",
|
|
" \"completion_tokens\": 13,\n",
|
|
" \"total_tokens\": 18830\n",
|
|
" }\n",
|
|
"}\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"# The response from original prompt, using GPT-4-32k\n",
|
|
"import json\n",
|
|
"prompt = \"\\n\\n\".join([instruction, contexts, question])\n",
|
|
"\n",
|
|
"message = [\n",
|
|
" {\"role\": \"user\", \"content\": prompt},\n",
|
|
"]\n",
|
|
"\n",
|
|
"request_data = {\n",
|
|
" \"messages\": message,\n",
|
|
" \"max_tokens\": 100,\n",
|
|
" \"temperature\": 0,\n",
|
|
" \"top_p\": 1,\n",
|
|
" \"n\": 1,\n",
|
|
" \"stream\": False,\n",
|
|
"}\n",
|
|
"response = openai.ChatCompletion.create(\n",
|
|
" \"gpt-4-32k\",\n",
|
|
" **request_data,\n",
|
|
")\n",
|
|
"print(json.dumps(response, indent=4))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "9aa90492-8ad1-4a89-85c5-26b8472f1ff0",
|
|
"metadata": {},
|
|
"source": [
|
|
"### The response of Compressed Prompt (Correct in 10x Compression)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 18,
|
|
"id": "fa638dec-c9ec-4dce-9dac-d768145de714",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"application/vnd.jupyter.widget-view+json": {
|
|
"model_id": "a935104a4dac4ffb882fcedb32dae917",
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
},
|
|
"text/plain": [
|
|
"Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s]"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"name": "stderr",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"/home/hjiang/.local/lib/python3.9/site-packages/transformers/generation/configuration_utils.py:362: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.9` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`. This was detected when initializing the generation config instance, which means the corresponding file may hold incorrect parameterization and should be fixed.\n",
|
|
" warnings.warn(\n",
|
|
"/home/hjiang/.local/lib/python3.9/site-packages/transformers/generation/configuration_utils.py:367: UserWarning: `do_sample` is set to `False`. However, `top_p` is set to `0.6` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `top_p`. This was detected when initializing the generation config instance, which means the corresponding file may hold incorrect parameterization and should be fixed.\n",
|
|
" warnings.warn(\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"# Setup LLMLingua\n",
|
|
"from llmlingua import PromptCompressor\n",
|
|
"llm_lingua = PromptCompressor()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 29,
|
|
"id": "12e3eb24-e195-42e6-8055-d7596f1fd78f",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"contexts_list = contexts.split(\"\\n\")\n",
|
|
"contexts_list = [\"\\n\".join(contexts_list[ii: ii + 4]) for ii in range(0, len(contexts_list), 4)]"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 30,
|
|
"id": "5f61a186-6641-4118-ad04-5245a53b6d79",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"{\n",
|
|
" \"compressed_prompt\": \"Please complete the code given below.\\n\\n };\\n}\\nSpectaculum-Demo/src/main/java/net/protyposis/android/spectaculumdemo/testeffect/InterlaceEffect.java\\npublic class InterlaceEffect ShaderEffect {\\n\\n return new TextureKernelShaderProgram(TextureKernelShaderProgram.Kernel.BLUR_GAUSS);\\n }\\n}\\nSpectaculum-Effect-FlowAbs/src/main/java/net/protyposis/android/spectaculum/effects/FlowAbsEffect.java\\n\\n void removeListener(Listener listener);\\n}\\nSpectaculum-Effect-Immersive/src/main/java/net/protyposis/android/spectaculum/effects/ImmersiveSensorNavigation.java\\npublic class ImmersiveSensorNavigation implements SensorEventListener {\\n\\n }\\n}\\nSpectaculum-Effect-FlowAbs/src/main/java/net/protyposis/android/spectaculum/effects/FlowAbsSubEffect.java\\npublic abstract class FlowAbsSubEffect extends BaseEffect {\\n\\n }\\n}\\nSpectaculum-Core/src/main/java/net/protyposis/android/spectaculum/effects/ContrastBrightnessAdjustmentEffect.java\\npublic class ContrastBrightnessAdjustmentEffect extends ShaderEffect {\\n\\n}\\nSpectaculum-Core/src/main/java/net/protyposis/android/spectaculum/effects/KernelEdgeDetectEffect.java\\npublic class KernelEdgeDetectEffect extends ShaderEffect {\\n @Override\\n\\n return new TextureKernelShaderProgram(TextureKernelShaderProgram.Kernel.SHARPEN);\\n }\\n}\\nSpectaculum-Core/src/main/java/net/protyposis/android/spectaculum/effects/ColorFilterEffect.java\\n\\n}\\nSpectaculum-Core/src/main/java/net/protyposis/android/spectaculum/effects/NoEffect.java\\npublic class NoEffect extends ShaderEffect {\\n\\n\\n}\\nSpectaculum-Core/src/main/java/net/protyposis/android/spectaculum/effects/Parameter.java\\npublic interface Parameter<T> {\\n\\n\\n}\\nSpectaculum-Core/src/main/java/net/protyposis/android/spectaculum/effects/StackEffect.java\\npublic class StackEffect extends BaseEffect {\\n\\n\\n }\\n}\\nSpectaculum-Core/src/main/java/net/protyposis/android/spectaculum/effects/SobelEffect.java\\npublic class SobelEffect extends ShaderEffect {\\n\\n }\\n}\\nSpectaculum-Core/src/main/java/net/protyposis/android/spectaculum/effects/KernelBlurEffect.java\\npublic class KernelBlurEffect extends ShaderEffect {\\n\\n}\\nSpectaculum-Core/src/main/java/net/protyposis/android/spectaculum/effects/KernelEmbossEffect.java\\npublic class KernelEmbossEffect extends ShaderEffect {\\n @Override\\n\\n}\\nSpectaculum-Core/src/main/java/net/protyposis/android/spectaculum/effects/Effect.java\\npublic interface Effect {\\n\\n\\nSpectaculum-Effect-Immersive/src/main/java/net/protyposis/android/spectaculum/effects/ImmersiveTouchNavigation.java\\npublic class ImmersiveTouchNavigation {\\n\\n private static final String TAG = ImmersiveTouchNavigation.class.getSimpleName();\\n\\nSpectaculum-Core/src/main/java/net/protyposis/android/spectaculum/effects/KernelGaussBlurEffect.java\\npublic class KernelGaussBlurEffect extends ShaderEffect {\\n @Override\\n protected TextureShaderProgram initShaderProgram() {\\n\\nSpectaculum-Effect-QrMarker/src/main/java/net/protyposis/android/spectaculum/effects/QrMarkerEffect.java\\npublic class QrMarkerEffect extends BaseEffect {\\n\\n private GaussShaderProgram mGaussShader;\\n\\n }\\n}\\nSpectaculum-Core/src/main/java/net/protyposis/android/spectaculum/effects/WatermarkEffect.java\\npublic class WatermarkEffect extends ShaderEffect {\\n\\n };}\\nSpectaculum-Core/src/main/java/net/protyposis/android/spectaculum/effects/FlipEffect.java\\npublic class FlipEffect extends ShaderEffect {\\n\\n}\\nSpectaculum-Core/src/main/java/net/protyposis/android/spectaculum/effects/SimpleToonEffect.java\\npublic class SimpleToonEffect extends ShaderEffect {\\n @Override\\n\\nSpectaculumCore/src/main/java/net/protyposis/android/spectaculum/effects/KernelSharpenEffect.java\\npublic class KernelSharpenEffect extends ShaderEffect {\\n @Override\\n protected TextureShaderProgram initShaderProgram() {\\n\\nsee net.protyposis.android.spectaculum.effects.Effect.\\n */\\n @Override\\n public void onAdded( effect Parameter parameter) {\\n * @see net.protyposis.android.spectaculum.effects.Effect.Listener\\n */\\n @Override\\n public void onParameterRemoved(Effect effect, Parameter parameter) {\\n\\n (!net.protyposis.android.spectacul.gles.GLUtilsisGEs2Supported {\\n Log.(TAG, \\\"GLES . is\\\"); return;\\n }SpectaculCore/src/main/java/net/protyposis/android/spectaculum/effects/EffectException.java\\npublic class EffectException extends Exception {() {\\nSpectacul-Core/src/main/java/net/protyposis/android/spectaculumgles/GLUtils.javapublic class GLUtils { final String TAG = GLUtils.class.getSimpleName();\\nSpectacul--Immers/src/main/java/net/protyposis/android/spectacul/effects/ImmersiveEffect.javapublic class Immersive Shader { /**\\n @see net.protyposis.android.spectacumsEffect.Listener \\n @Override\\n public void onChanged( effect)Spectacul/src/main/java/net/protyposis/android/spectacul/SpectaculumView\\n class SpectaculumLSurfaceView\\n SurfaceTexture.OnFrameAvailableListener return (FlowAbNoiseEffect FlowAbNoiseTexture().);\\n\\n\\n QrMarker {\\n anny Canny();\\n }\\n\\n if(!mFlowAbsEffect.ized\\nFlowAbsEffect.init,); }\\n \\n @Override\\n public() { SpectacumthisonEffectErrorindex, effect e); } mSpectaculum =;\\n ector =ureDetector(mSpectaculumViewContext mOnListener);\\n\\n public void setEffectEventListener(Effect listener mEffectEventListener = listener; \\n\\n\\n @Override\\n run() SpectaculumViewthis.onEffectInitialized(index, effect);\\n SpectaculViewthis.onEffectSelected,\\n }\\n\\n Effect.Listener, GLRenderer.EffectEventListener,\\n GLRenderer.OnFrameCapturedCallback {\\n\\n private static final String TAG = SpectaculumView.class.getSimpleName();\\n\\n QrMarkerEffect.this.init(width, height);\\n }\\n }\\n\\n\\n\\n public interface EffectEventListener extends GLRenderer.EffectEventListener {}\\n public interface OnFrameCapturedCallback extends GLRenderer.OnFrameCapturedCallback {}\\n\\n\\n\\n surfaceTexture.setOnFrameAvailableListener(SpectaculumView.this);\\n }\\n };\\n\\n mEffectEventListener.onEffectError(index, effect, e);\\n }\\n }\\n\\n\\nimport android.app.Activity;\\nimport android.graphics.BitmapFactory;\\nimport android.view.Menu;\\nimport android.view.MenuItem;\\nimport android.view.SubMenu;\\nimport android.view.View;\\nimport android.view.ViewGroup;\\nimport android.widget.Toast;\\nimport java.util.ArrayList;\\nimport java.util.List;\\nimport net.protyposis.android.spectaculum.SpectaculumView;\\nimport net.protyposis.android.spectaculum.effects.Parameter;\\nimport net.protyposis.android.spectaculum.effects.ImmersiveSensorNavigation;\\nimport net.protyposis.android.spectaculum.effects.ContrastBrightnessAdjustmentEffect;\\nimport net.protyposis.android.spectaculum.effects.EffectException;\\nimport net.protyposis.android.spectaculum.effects.FlowAbsSubEffect;\\nimport net.protyposis.android.spectaculum.effects.QrMarkerEffect;\\nimport net.protyposis.android.spectaculum.effects.Effect;\\nimport net.protyposis.android.spectaculum.effects.FlipEffect;\\nimport net.protyposis.android.spectaculum.effects.FlowAbsEffect;\\nimport net.protyposis.android.spectaculum.effects.KernelBlurEffect;\\nimport net.protyposis.android.spectaculum.effects.KernelEdgeDetectEffect;\\nimport net.protyposis.android.spectaculum.effects.KernelEmbossEffect;\\nimport net.protyposis.android.spectaculum.effects.KernelGaussBlurEffect;\\nimport net.protyposis.android.spectaculum.effects.KernelSharpenEffect;\\nimport net.protyposis.android.spectaculum.effects.NoEffect;\\nimport net.protyposis.android.spectaculum.effects.SimpleToonEffect;\\nimport net.protyposis.android.spectaculum.effects.SobelEffect;\\nimport net.protyposis.android.spectaculum.effects.ImmersiveTouchNavigation;\\nimport net.protyposis.android.spectaculum.effects.StackEffect;\\nimport net.protyposis.android.spectaculum.effects.WatermarkEffect;\\nimport net.protyposis.android.spectaculum.gles.GLUtils;\\nimport net.protyposis.android.spectaculum.effects.ColorFilterEffect;\\nimport net.protyposis.android.spectaculum.effects.ImmersiveEffect;\\nimport net.protyposis.android.spectaculumdemo.testeffect.InterlaceEffect;\\n/*\\n * Copyright 2014 Mario Guggenberger <mg@protyposis.net>\\n *\\n * Licensed under the Apache License, Version 2.0 (the \\\"License\\\");\\n * you may not use this file except in compliance with the License.\\n * You may obtain a copy of the License at\\n *\\n * http://www.apache.org/licenses/LICENSE-2.0\\n *\\n * Unless required by applicable law or agreed to in writing, software\\n * distributed under the License is distributed on an \\\"AS IS\\\" BASIS,\\n * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\\n * See the License for the specific language governing permissions and\\n * limitations under the License.\\n */\\n\\npackage net.protyposis.android.spectaculumdemo;\\n\\n\\n\\n\\n/**\\n * Created by Mario on 18.07.2014.\\n *\\n * Helper class for easy effect handling in the various Spectaculum views in this demo.\\n * Provides a list of effects for the actionbar and displays a parameter control panel for\\n * selected effects with parameters that the demo user can player play with.\\n */\\npublic class EffectManager implements SpectaculumView.EffectEventListener, Effect.Listener {\\n\\n private Activity mActivity;\\n private ViewGroup mParameterListView;\\n private EffectParameterListAdapter mParameterListAdapter;\\n private MenuItem mParameterToggleMenuItem;\\n private SpectaculumView mSpectaculumView;\\n private List<Effect> mEffects;\\n private Effect mSelectedEffect;\\n private ImmersiveSensorNavigation mImmersiveSensorNavigation;\\n private ImmersiveTouchNavigation mImmersiveTouchNavigation;\\n\\n public EffectManager(Activity activity, int parameterListViewId, SpectaculumView glView) {\\n mActivity = activity;\\n mParameterListView = (ViewGroup) activity.findViewById(parameterListViewId);\\n mSpectaculumView = glView;\\n mEffects = new ArrayList<>();\\n mSpectaculumView.setEffectEventListener(this);\\n\\n // Spectaculum-Core filters\\n\\nNext line of code:\\n\",\n",
|
|
" \"origin_tokens\": 18837,\n",
|
|
" \"compressed_tokens\": 2281,\n",
|
|
" \"ratio\": \"8.3x\",\n",
|
|
" \"saving\": \", Saving $1.0 in GPT-4.\"\n",
|
|
"}\n",
|
|
"Response: {\n",
|
|
" \"id\": \"chatcmpl-8Fk3A6V1ecGT9ixENp4JVpAFqMX3q\",\n",
|
|
" \"object\": \"chat.completion\",\n",
|
|
" \"created\": 1698762628,\n",
|
|
" \"model\": \"gpt-4-32k\",\n",
|
|
" \"choices\": [\n",
|
|
" {\n",
|
|
" \"index\": 0,\n",
|
|
" \"finish_reason\": \"stop\",\n",
|
|
" \"message\": {\n",
|
|
" \"role\": \"assistant\",\n",
|
|
" \"content\": \"mEffects.add(new NoEffect());\"\n",
|
|
" }\n",
|
|
" }\n",
|
|
" ],\n",
|
|
" \"usage\": {\n",
|
|
" \"prompt_tokens\": 2288,\n",
|
|
" \"completion_tokens\": 7,\n",
|
|
" \"total_tokens\": 2295\n",
|
|
" }\n",
|
|
"}\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"# 8x Compression\n",
|
|
"compressed_prompt = llm_lingua.compress_prompt(\n",
|
|
" contexts_list,\n",
|
|
" instruction=instruction,\n",
|
|
" question=question,\n",
|
|
" target_token=2000,\n",
|
|
" condition_compare=True,\n",
|
|
" condition_in_question='after',\n",
|
|
" rank_method='longllmlingua',\n",
|
|
" use_sentence_level_filter=False,\n",
|
|
" context_budget=\"+100\",\n",
|
|
" dynamic_context_compression_ratio=0.4, # enable dynamic_context_compression_ratio\n",
|
|
" reorder_context=\"sort\"\n",
|
|
")\n",
|
|
"message = [\n",
|
|
" {\"role\": \"user\", \"content\": compressed_prompt[\"compressed_prompt\"]},\n",
|
|
"]\n",
|
|
"\n",
|
|
"request_data = {\n",
|
|
" \"messages\": message,\n",
|
|
" \"max_tokens\": 100,\n",
|
|
" \"temperature\": 0,\n",
|
|
" \"top_p\": 1,\n",
|
|
" \"n\": 1,\n",
|
|
" \"stream\": False,\n",
|
|
"}\n",
|
|
"response = openai.ChatCompletion.create(\n",
|
|
" \"gpt-4-32k\",\n",
|
|
" **request_data,\n",
|
|
")\n",
|
|
"\n",
|
|
"print(json.dumps(compressed_prompt, indent=4))\n",
|
|
"print(\"Response:\", response)"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3 (ipykernel)",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.9.18"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|