# Keyword Counting

The use case in this directory computes the frequencies of occurring countries 
in a long passage of text. We provide implementations of seven different approaches:
- IO
- Chain-of-Thought (CoT)
- Tree of Thought (ToT):
  - ToT: wider tree, meaning more branches per level
  - ToT2: tree with more levels, but fewer branches per level
- Graph of Thoughts (GoT):
  - GoT4: split passage into 4 sub-passages
  - GoT8: split passage into 8 sub-passages
  - GoTx: split by sentences

## Data

We provide an input file with 100 samples: `countries.csv`. It is also possible to use
the data generator `dataset_gen_countries.py` to generate additional or
different samples (using GPT-4). The parameters can be updated on line 54 (number of samples to be generated). 
Note that not every generated sample will be included in the dataset, as each sample is 
additionally tested for validity (observe script output for details).

## Execution

The file to execute the use case is called
`keyword_counting.py`. In the main body, one can
select the specific samples to be run (variable samples) and the
approaches (variable approaches). It is also possible to set a budget in
dollars (variable budget).

The Python scripts will create the directory `result`, if it is not
already present. In the `result` directory, another directory is created
for each run: `{name of LLM}_{list of approaches}_{day}_{start time}`.
Inside each execution specific directory two files (`config.json`,
`log.log`) and a separate directory for each selected approach are
created. `config.json` contains the configuration of the run: input data,
selected approaches, name of the LLM, and the budget. `log.log` contains
the prompts and responses of the LLM as well as additional debug data.
The approach directories contain a separate json file for every sample
and the file contains the Graph Reasoning State (GRS) for that sample.

## Plot Data

Change the results directory in line 150 of `plot.py` and run `python3
plot.py` to plot your data.