Updates on FinGPT Sentiment Analysis v1

fingpt/FinGPT_Sentiment_Analysis_v1/FinGPT_v1.1/README.md

@@ -0,0 +1,102 @@
+# FinGPT-V1.1
+
+## Ⅰ. Results & Findings
+
+|             Metrics$^{[1]}$             | Llama2-chat | Ours  | Improvement |
+| :-------------------------------------: | :---------: | :---: | :---------: |
+| Accuracy / Micro f1 (7 classes$^{[2]}$) |    12.61    | 14.59 |   15.7 %    |
+|     Weighted F1 (7 classes$^{[2]}$)     |    4.17     | 11.11 |    166 %    |
+|      Macro F1 (7 classse$^{[2]}$)       |    0.16     | 9.23  |   1000+ %   |
+| Accuracy / Micro f1 (3 classes$^{[3]}$) |    26.85    | 41.35 |    54 %     |
+|     Weighted F1 (3 classes$^{[3]}$)     |    14.29    | 35.67 |   149.6 %   |
+|      Macro F1 (3 classes$^{[3]}$)       |    12.69    | 22.73 |   79.12 %   |
+
+$^{[1]}$ The groud-true label for the result is generated from the market, please refer to 2.3 section.
+$^{[2]}$ 7 classse means the result is one of `Severely Negative`, `Moderately Negative`, `Mildly Negative`, `Neutral`, `Mildly Positive`, `Moderately Positive`, `Severely Positive`. 
+$^{[3]}$For 3 classes, `Severely Negative` and `Moderately Negative` are considered `Negative`; `Mildly Negative`,  `Neutral` and `Mildly Positive` are considered `Neutral`; `Severely Positive`and `Moderately Positive` are considered `Positive`
+
+* The analysis of LLM itself might not align with the market, but we are able to finetune our model to make it align with the market.
+
+## Ⅱ. Data
+
+### 2.1 Data overview (news)
+
+* The data are gathered from online open data sources with exact timestamp. 
+
+* It was split into the training and testing period as follow:
+
+  ``` 
+  train_start_date = '2019-01-01'
+  train_end_date = '2021-12-31'
+  test_start_date = '2022-01-01'
+  test_end_date = '2023-08-31'
+  ```
+
+### 2.2 Data Aggregation
+
+* To make things better, the best way is to use the new title with the new content. However, it would exceed the max length of 4096, so we get rid of the parts that are too long
+* The **News Title** of the news was selected to shorten the total length in order to take more news into consideration.
+
+### 2.3 Label Generation
+
+* The Label was set according to **5-day price change(FDPC)**, it follows the rules below:
+
+  | (-∞, -0.06)       | [-0.06, -0.03)      | [-0.03, -0.01)  | [-0.01, 0.01) | [0.01, 0.03)    | [0.03, 0.06)        | [0.06, +∞)        |
+  | ----------------- | ------------------- | --------------- | ------------- | --------------- | ------------------- | ----------------- |
+  | Severely Negative | Moderately Negative | Mildly Negative | Neutral       | Mildly Positive | Moderately Positive | Severely Positive |
+
+## Ⅲ. Experiment setting
+
+* Model setting:
+
+  ``` python
+  model_name = "daryl149/llama-2-13b-chat-hf"  
+  tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
+  tokenizer.pad_token_id = tokenizer.eos_token_id
+  model =  AutoModelForCausalLM.from_pretrained(
+          model_name, 
+          trust_remote_code=True, 
+          device_map='auto',
+      )
+  ```
+
+* Training args:
+
+   ``` python
+   training_args = TrainingArguments(
+           output_dir='./finetuned_model',    # saved model path
+           logging_steps = 500,
+           # max_steps=10000,
+           num_train_epochs = 2,
+           per_device_train_batch_size=4,
+           gradient_accumulation_steps=8,
+           learning_rate=1e-4,
+           weight_decay=0.01,
+           warmup_steps=1000,
+           save_steps=500,
+           fp16=True,
+           # bf16=True,
+           torch_compile = False,
+           load_best_model_at_end = True,
+           evaluation_strategy="steps",
+           remove_unused_columns=False,
+   )
+   ```
+
+* LoRA args:
+
+   ``` python
+   target_modules = TRANSFORMERS_MODELS_TO_LORA_TARGET_MODULES_MAPPING['llama']
+   lora_config = LoraConfig(
+       task_type=TaskType.CAUSAL_LM,
+       inference_mode=False,
+       r=8,
+       lora_alpha=32,
+       lora_dropout=0.1,
+       target_modules=target_modules,
+       bias='none',
+   )
+   ```
+
+   
+

fingpt/FinGPT_Sentiment_Analysis_v1/README.md

@@ -0,0 +1,60 @@
+# FinGPT-V1 series (Labeled by the Market)
+
+## You can try our FinGPT v1.1 model as follows or [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1ES-_w0iUGIQJnIGbXjU_c1ei8BsTi_uO?usp=sharing):
+
+``` python
+from peft import PeftModel, PeftConfig
+from transformers import LlamaForCausalLM, LlamaTokenizerFast
+
+# Load Models
+base_model = "daryl149/llama-2-13b-chat-hf" 
+peft_model = "oliverwang15/FinGPT_v11_Llama2_13B_Sentiment_Market_Feedback_LoRA_FT_8bit"
+tokenizer = LlamaTokenizerFast.from_pretrained(base_model, trust_remote_code=True)
+tokenizer.pad_token = tokenizer.eos_token
+model = LlamaForCausalLM.from_pretrained(base_model, trust_remote_code=True, device_map = "cuda:0", load_in_8bit = True,)
+model = PeftModel.from_pretrained(model, peft_model)
+model = model.eval()
+
+# Make prompts
+# Please change the news to the news you want
+news = 'A tinyurl link takes users to a scamming site promising that users can earn thousands of dollars by becoming a Google ( NASDAQ : GOOG ) Cash advertiser'
+template = """News: '''{}'''
+
+Instruction: Please 'ONLY' output 'one' sentiment of all the above News from {{ Severely Positive / Moderately Positive / Mildly Positive / Neutral / Mildly Negative / Moderately Negative / Severely Negative }} without other words.
+
+Answer:
+"""
+prompt = template.format(mews)
+
+# Generate results
+tokens = tokenizer(prompt, return_tensors='pt', padding=True)
+tokens = tokens.to(model.device)
+with torch.no_grad():
+  res = model.generate(**tokens, max_length=1024)
+torch.cuda.empty_cache()
+res_sentences = [tokenizer.decode(i, skip_special_tokens=True) for i in res]
+out_text = [o.split("Answer:")[1] for o in res_sentences]
+sentiment = out_text[0].strip()
+print(sentiment)
+# Severely Negative
+```
+
+
+
+## Ⅰ. What is FinGPT-V1 series
+
+* FinGPT-V1 is a series of LoRA model for financial sentiment analysis whose labels are generated by the change of market (stock price). 
+* You can also say these models are analyzing or predicting the market through related alternative data (news, social media, filings, etc,).
+
+## Ⅱ. Why label by the market
+
+* Labeling has always been a big issue in the ever-changing financial market, labeling by the market is more efficient and economic than human labor (not necessarily better).
+* Human's opinion on the market is sometimes biased. Since our final goal is to analyze or predict the market, why not let the model learn directly from the market.
+
+### Ⅲ. What do we have
+
+| Project/Model              | Base Model    | Data Type | Market | Frequency | # of Emotions |
+| -------------------------- | ------------- | --------- | ------ | --------- | ------------- |
+| [FinGPT v1.0](./FinGPT_v1.0/README.md) | ChatGLM2-6B   | News      | CN     | 1 day     | 5             |
+| [FinGPT v1.1](./FinGPT_v1.1/README.md) | Llama 2 - 13B | News      | US     | 1 day     | 7             |
+

fingpt/FinGPT_v1/README.md

@@ -1,19 +0,0 @@
-# FinGPT-V1 series (Labeled by the Market)
-
-## Ⅰ. What is FinGPT-V1 series
-
-* FinGPT-V1 is a series of LoRA model for financial sentiment analysis whose labels are generated by the change of market (stock price). 
-* You can also say these models are analyzing or predicting the market through related alternative data (news, social media, filings, etc,).
-
-## Ⅱ. Why label by the market
-
-* Labeling has always been a big issue in the ever-changing financial market, labeling by the market is more efficient and economic than human labor (not necessarily better).
-* Human's opinion on the market is sometimes biased. Since our final goal is to analyze or predict the market, why not let the model learn directly from the market.
-
-### Ⅲ. What do we have
-
-| Project/Model              | Base Model    | Data Type | Market | Frequency | # of Emotions |
-| -------------------------- | ------------- | --------- | ------ | --------- | ------------- |
-| [FinGPT v1.0](./FinGPT_v1.0/README.md) | ChatGLM2-6B   | News      | CN     | 1 day     | 5             |
-| [FinGPT v1.1](./FinGPT_v1.1/README.md) | Llama 2 - 13B | News      | US     | 1 day     | 7             |
-

fingpt/readme.md

@@ -1,12 +1,15 @@
 # FinGPT's playground
 
 ## Ⅰ. FinGPT Models
-* [FinGPT-Forecaster](https://github.com/AI4Finance-Foundation/FinGPT/tree/master/fingpt/FinGPT_Forecaster): FinGPT for Company News Analysis and Stock Price Prediction
 * [FinGPT-Benchmark](https://github.com/AI4Finance-Foundation/FinGPT/tree/master/fingpt/FinGPT_Benchmark): FinGPT Instruction Tuning paradigm for task-specific, multi-task, and zero-shot instruction tuning tasks
+* [FinGPT-Forecaster](https://github.com/AI4Finance-Foundation/FinGPT/tree/master/fingpt/FinGPT_Forecaster): FinGPT for Company News Analysis and Stock Price Prediction
+* [FinGPT-Low-Code-Development](https://github.com/AI4Finance-Foundation/FinGPT/tree/master/fingpt/FinGPT_Low_Code_Development): FinGPT for low-code development in Financial Field
 * [FinGPT-RAG](https://github.com/AI4Finance-Foundation/FinGPT/tree/master/fingpt/FinGPT_RAG): FinGPT using Retrieval Augmented Generation pipeline
-* [FinGPT-v1](https://github.com/AI4Finance-Foundation/FinGPT/tree/master/fingpt/FinGPT_v1): FinGPT from Market labels in Chinese Market
-* [FinGPT-v2](https://github.com/AI4Finance-Foundation/FinGPT/tree/master/fingpt/FinGPT_v2): FinGPT building POC using ChatGPT, such as Robo-advisor, low-code development, and trading
-* [FinGPT-v3](https://github.com/AI4Finance-Foundation/FinGPT/tree/master/fingpt/FinGPT_v3): FinGPT for Sentiment Analysis in U.S. Market
-
+* [FinGPT-Robo-Advisor](https://github.com/AI4Finance-Foundation/FinGPT/tree/master/fingpt/FinGPT_Robo-Advisor)
+* [FinGPT-Sentiment-Analysis-v1](https://github.com/AI4Finance-Foundation/FinGPT/tree/master/fingpt/FinGPT_Sentiment_Analysis_v1): FinGPT for Sentiment Analysis from Market labels
+* [FinGPT-Sentiment-Analysis-v2](https://github.com/AI4Finance-Foundation/FinGPT/tree/master/fingpt/FinGPT_Sentiment_Analysis_v2): FinGPT for Sentiment Analysis from LLM labels
+* [FinGPT-Sentiment-Analysis-v3](https://github.com/AI4Finance-Foundation/FinGPT/tree/master/fingpt/FinGPT_Sentiment_Analysis_v3): FinGPT for Sentiment Analysis from Academic datasets to showcase the power of LoRA
+* [FinGPT-Trading](https://github.com/AI4Finance-Foundation/FinGPT/tree/master/fingpt/FinGPT_Trading): Possible ways to apply LLMs in practical tradings
+* [FinGPT-Others](https://github.com/AI4Finance-Foundation/FinGPT/tree/master/fingpt/FinGPT_Others): Other FinGPT codes