fukkatsu

Dynamic Software Improvement and Mutation using LLMs for Stochastic Synthetic Code Injections

https://github.com/maxmekiska/fukkatsu

Keywords

code-correction code-mutation llm

Last synced: 7 months ago · JSON representation ·

Repository

Dynamic Software Improvement and Mutation using LLMs for Stochastic Synthetic Code Injections

Basic Info

Host: GitHub
Owner: maxmekiska
License: mit
Language: Python
Default Branch: main
Homepage:
Size: 1.62 MB

Statistics

Stars: 3
Watchers: 1
Forks: 3
Open Issues: 0
Releases: 13

Topics

code-correction code-mutation llm

Created almost 3 years ago · Last pushed over 1 year ago

Metadata Files

Readme Changelog License Citation

fukkatsu

| Build | Status| |---|---| | MAIN BUILD | | | DEV BUILD | |

pip install fukkatsu

API Keys

fukkatsu requires the environmental variable OPENAI_API_KEY set.

Description

This is a proof of concept for a library that will leverage LLMs to dynamically fix and improve code during execution. fukkatsu is the japanese word, 復活, for "resurrection" or "revival". Metaphorically speaking, this library will attempt to fix your cars tire while you are driving it at 300 km/h.

This concept currently only applies to interpreted languages such as python and not to compiled languages such as C++. The very nature of interpreted languages allows us to dynamically change the code during runtime.

Furthermore, fukkatsu introduces a method to enhance ordinary functions with the power of LLMs. By decorating ordinary functions with natural language prompts, they can now dynamically adapt to unforeseen inputs.

Medium post

fukkatsu — Dynamic Software Improvement and Mutation using LLMs for Stochastic Synthetic Code Injections

Quick Start

```python import pandas as pd from datetime import datetime

from fukkatsu import resurrect

@resurrect( lives=3, allowinstalls = True, additionalreq = "Account for multiple date formats if necessary.", activetwin = True, primarymodelapi = "gateway", secondarymodelapi = "gateway", primaryconfig = {"temperature": 0.01, "model": "meta-llama/llama-3.1-8b-instruct:free"}, secondaryconfig = {"temperature": 0.45, "model": "meta-llama/llama-3.1-8b-instruct:free"}, ) def performdatatransformation(data): """takes in list of date strings and transforms them into datetime objects. """ dateformat = '%Y-%m-%d'

for idx, date in enumerate(data):
    data[idx] = datetime.strptime(date, date_format)

return data

if name == "main":

data = [ "2023-07-07", "1 June 2020", "2023.07.07", "2023-12-01", "2020/01/01", "Nov 11 1994" ]

transformeddata = performdata_transformation(data)

transformed_data ```

fukkatsu 0.0.1 - `Extra Life`

Expand

fukkatsu 0.0.1 incorporates all the features demonstrated within the MVP section and introduces the concept of additional requests. Additional requests provide users with an alternative means of giving specific instructions to the LLM when a correction to a function is required. These additional requests act as a safeguard against potential misinterpretations by the LLM. ```python @resurrect(lives=1, additional_req = "add to any result 1000") def my_function(x, y, z): """ function to divide x by y and add to the result z. Should return z if y is 0. """ result = x / y + z return result print(my_function(x = 1, y = 0, z= 2)) print(my_function(x = 1, y = 0, z= 2)) # second function will trigger short term memory capabilities ``` ``` ERROR:root:division by zero Traceback (most recent call last): File "xxxxxxxxxxxxxxxxxxxxx", line 20, in wrapper result = func(*args, **kwargs) File "xxxxxxxxxxxxxxxxxxxxx", line 6, in my_function result = x / y + z ZeroDivisionError: division by zero WARNING:root:Input arguments: {'x': 1, 'y': 0, 'z': 2} WARNING:root: Source Code: def my_function(x, y, z): """ function to divide x by y and add to the result z. Should return z if y is 0. """ result = x / y + z return result WARNING:root:Requesting INITIAL correction WARNING:root:Received INITIAL suggestion: def my_function(x, y, z): """ function to divide x by y and add to the result z. Should return z if y is 0. """ if y == 0: return z + 1000 else: result = x / y + z return result + 1000 WARNING:root:Attempt 1 to reanimate WARNING:root:Reanimation successful, using def my_function(x, y, z): """ function to divide x by y and add to the result z. Should return z if y is 0. """ if y == 0: return z + 1000 else: result = x / y + z return result + 1000 ERROR:root:division by zero Traceback (most recent call last): File "xxxxxxxxxxxxxxxxxxxxxxx", line 20, in wrapper result = func(*args, **kwargs) File "xxxxxxxxxxxxxxxxxxxxxxx", line 6, in my_function result = x / y + z ZeroDivisionError: division by zero WARNING:root:Input arguments: {'x': 1, 'y': 0, 'z': 2} WARNING:root: Source Code: def my_function(x, y, z): """ function to divide x by y and add to the result z. Should return z if y is 0. """ result = x / y + z return result WARNING:root:Correction already in memory WARNING:root:Attempt 1 to reanimate WARNING:root:Reanimation successful, using def my_function(x, y, z): """ function to divide x by y and add to the result z. Should return z if y is 0. """ if y == 0: return z + 1000 else: result = x / y + z return result + 1000 ``` ``` 1002 1002 ```

fukkatsu 0.0.2 - `The Ghost in the Machine`

Expand

The `mutate` decorator introduces a new way to enhance ordinary functions dynamically via the power of LLMs, enabling them to adapt to specific inputs. It provides users with the ability to extend the capabilities of functions through natural language prompts. Additionally, the decorator can be further extended using the `resurrect` decorator. The `mutate` decorator enables users to program and account for cases that are challenging or impossible to anticipate. ```python @resurrect(lives=1) @mutate(request= "Check the inputs closely. Given the inputs, make sure that the function is able to handle different formats if neccessary") def my_mutated_function(file_path: str) -> pd.DataFrame(): """ function to read files and output a dataframes. """ pd.read_csv(file_path) my_mutated_function("test_file.xlsx") ```

fukkatsu 0.0.3 - `Laissez-faire`

Expand

The `mutate` and `resurrect` decorators now support a new argument called allow_installs. By default, `allow_installs` is set to `False`. However, when set to `True`, the LLM will be able to test whether suggested or used python libraries are installed on the system. If any of the libraries are not installed, the LLM will install them before continuing code execution. This argument enables the LLM to have even more freedom. Therefore, setting the argument to True should be considered carefully. ### `resurrect` ```python def resurrect(lives: int = 1, additional_req: str = "", allow_installs: bool = False): ... ``` ### `mutate` ```python def mutate(request: str = "", allow_installs: bool = False): ... ```

fukkatsu 0.0.5 - `Not so Evil Twin`

Expand

The `mutate` and `resurrect` decorators now support new arguments `active_twin`, `llm`, and `temperature`. By default, `active_twin` is set to `False`, `llm` is set to `{"primary": "gpt-3.5-turbo", "secondary": "gpt-3.5-turbo"}`, and `temperature` is set to `{"primary": 0.1, "secondary": 0.1}`. This allows the user to configure the two decorators in a more granular way. If `active_twin` is set to `True`, another LLM, the `TWIN`, will crosscheck the answer of the first LLM and make corrections if deemed necessary. This is highly experimental but might become very powerful as soon as more diverse LLMs become available. ### `resurrect` ```python def resurrect( lives: int = 1, additional_req: str = "", allow_installs: bool = False, active_twin: bool = False, llm: dict = {"primary": "gpt-3.5-turbo", "secondary": "gpt-3.5-turbo"}, temperature: dict = {"primary": 0.1, "secondary": 0.1}, ): ... ``` ### `mutate` ```python def mutate( request: str = "", allow_installs: bool = False, active_twin: bool = False, llm: dict = {"primary": "gpt-3.5-turbo", "secondary": "gpt-3.5-turbo"}, temperature: dict = {"primary": 0.1, "secondary": 0.1}, ): ... ```

fukkatsu 0.0.8 - `I can see you`

Expand

This release features a new decorator called `stalk`. The `stalk` decorator enables you to quality-check your functions during runtime. Stalk will randomly execute when your target function is called. The primary objective is to check if your target functions are still working as intended during your program execution. If stalk deems your function as behaving illogically, stalk will perform modifications and enhancements similar to the `mutate` decorator. You can decide how frequent stalk will check a particular function by setting the likelihood parameter. By default, the likelihood parameter is set to 1. A value of 1 indicates that stalk will quality-check the function every time it is called. A value of 0.5 indicates that stalk will quality-check the function half of the time it is called. ### `stalk` ```python def stalk( likelihood: float = 1, additional_req: str = "", allow_installs: bool = False, active_twin: bool = False, llm: dict = {"primary": "gpt-3.5-turbo", "secondary": "gpt-3.5-turbo"}, temperature: dict = {"primary": 0.1, "secondary": 0.1}, ): ... ```

fukkatsu 0.0.10 - `Sharing is Caring`

Expand

This release includes new updates to the three decorators: `resurrect`, `mutate`, and `stalk`. Each decorator is now ready to support language model providers other than OpenAI in the future. To enable this, various changes have been made to the arguments. Please see below for the new arguments. By default, all models will be set to OpenAI. Support for new providers will be added as soon as they become available. Configurating the `openai` model API via: ```python @dataclass class OpenaiChatCompletionConfig: model: str temperature: float max_tokens: int n: int stop: Optional[str] ``` The default values set for the `openai` model API: ```python model: str = "gpt-3.5-turbo", temperature: float = 0.1, max_tokens: int = 1024, n: int = 1, stop: str = None, ``` ### `resurrect` ```python def resurrect( lives: int = 1, additional_req: str = "", allow_installs: bool = False, active_twin: bool = False, primary_model_api: str = "openai", secondary_model_api: str = "openai", primary_config: dict = {}, secondary_config: dict = {}, ): ... ``` ### `mutate` ```python def mutate( request: str = "", allow_installs: bool = False, active_twin: bool = False, primary_model_api: str = "openai", secondary_model_api: str = "openai", primary_config: dict = {}, secondary_config: dict = {}, ): ... ``` ### `stalk` ```python def stalk( likelihood: float = 1.0, additional_req: str = "", allow_installs: bool = False, active_twin: bool = False, primary_model_api: str = "openai", secondary_model_api: str = "openai", primary_config: dict = {}, secondary_config: dict = {}, ): ... ``` ### Appendix: How to use fukkatsu in a python class? fukkatsu wrappers can be used in python classes in the following way: ```python from typing import List import pandas as pd from datetime import datetime from fukkatsu import resurrect, mutate, stalk, reset_openai_key @resurrect( lives=3, allow_installs = True, additional_req = "Account for multiple dateformats if necessary.", active_twin = True, primary_model_api = "openai", secondary_model_api = "openai", primary_config = {"model": "gpt-3.5-turbo", "temperature": 0.88}, secondary_config = {"model": "gpt-3.5-turbo", "temperature": 0.33} ) def perform_data_transformation(data:list): """takes in list of datestrings, transforms into datetime objects. """ date_format = '%Y-%m-%d' for idx, date in enumerate(data): data[idx] = datetime.strptime(date, date_format) return data data = ["2023-07-07", "1 June 2020", "2023.07.07", "2023-12-01", "2020/01/01", "Nov 11 1994"] class TestClass: def __init__(self): self.test = "test" def test_wrapper_in_class(self, data: List): return perform_data_transformation(data) test = TestClass() test.test_wrapper_in_class(data) ```

fukkatsu 0.0.11 - `The Humans are back`

Expand

Feature to get human-in-the-loop functionality. Once a successful correction was determind, the user will be asked to confirm the correction suggestion via a simple "y" or "n" command line input. ### `resurrect` ```python def resurrect( lives: int = 1, additional_req: str = "", allow_installs: bool = False, active_twin: bool = False, primary_model_api: str = "openai", secondary_model_api: str = "openai", primary_config: dict = {}, secondary_config: dict = {}, human_action: bool = False, active_memory: bool = True, ): ... ``` ### `mutate` ```python def mutate( request: str = "", allow_installs: bool = False, active_twin: bool = False, primary_model_api: str = "openai", secondary_model_api: str = "openai", primary_config: dict = {}, secondary_config: dict = {}, human_action: bool = False, ): ... ``` ### `stalk` ```python def stalk( likelihood: float = 1.0, additional_req: str = "", allow_installs: bool = False, active_twin: bool = False, primary_model_api: str = "openai", secondary_model_api: str = "openai", primary_config: dict = {}, secondary_config: dict = {}, human_action: bool = False, ): ... ``` ## Appendix Added `active_memory` parameter to control the activation of the short term memory. Setting the `active_memory` parameter to `False` will prevent the `resurrect` decorator from remembering past solutions.

fukkatsu 0.0.13 - `Making new Friends`

Expand

This release will support Google's `gemini-pro` LLM. Each decorator will now support the google generative-ai SDK. The following shows an example configuration that leverages OpenAI and Google LLM's: ### `resurrect` ```python def resurrect( lives: int = 1, additional_req: str = "", allow_installs: bool = False, active_twin: bool = True, primary_model_api: str = "openai", secondary_model_api: str = "google", primary_config = {"model": "gemini-pro", "temperature": 0.1}, secondary_config = {"model": "gpt-3.5-turbo", "temperature": 0.1}, human_action: bool = True, active_memory: bool = True, ): ... ``` ### Example ressurection configuration ```python import fukkatsu print(fukkatsu.__version__) from fukkatsu import resurrect import pandas as pd from datetime import datetime @resurrect( lives=3, allow_installs = True, additional_req = "Account for multiple dateformats if necessary.", active_twin = True, primary_model_api = "google", secondary_model_api = "openai", primary_config = {"model": "gemini-pro", "temperature": 0.1}, secondary_config = {"model": "gpt-3.5-turbo", "temperature": 0.1}, human_action = True, active_memory = True ) def perform_data_transformation(data): """takes in list of datestrings, transforms into datetime objects. """ date_format = '%Y-%m-%d' for idx, date in enumerate(data): data[idx] = datetime.strptime(date, date_format) return data data = [ "2023-07-07", "1 June 2020", "2023.07.07", "2023-12-01", "2020/01/01", "Nov 11 1994" ] transformed_data = perform_data_transformation(data) print(transformed_data) ``` ### Example logs of a live resurrection - Twin mode OpenAI + Google ``` (env) PS C:\Users\Max\Documents\Misc\fukkatsu-integration-tests> python .\test-dates-twin.py 2023-12-20 01:42:13,337 - Setting OPENAI_API_KEY 2023-12-20 01:42:13,337 - OPENAI_API_KEY found in environment variables. 2023-12-20 01:42:13,337 - Setting GOOGLE_API_KEY 2023-12-20 01:42:13,337 - GOOGLE_API_KEY found in environment variables. 2023-12-20 01:42:13,866 - time data '1 June 2020' does not match format '%Y-%m-%d' Traceback (most recent call last): File "c:\users\max\documents\research\fukkatsu\fukkatsu\fukkatsu\__init__.py", line 43, in wrapper result = func(*args_copy, **kwargs_copy) File "C:\Users\Max\Documents\Misc\fukkatsu-integration-tests\test-dates-twin.py", line 29, in perform_data_transformation data[idx] = datetime.strptime(date, date_format) File "C:\Users\Max\AppData\Local\Programs\Python\Python39\lib\_strptime.py", line 568, in _strptime_datetime tt, fraction, gmtoff_fraction = _strptime(data_string, format) File "C:\Users\Max\AppData\Local\Programs\Python\Python39\lib\_strptime.py", line 349, in _strptime raise ValueError("time data %r does not match format %r" % ValueError: time data '1 June 2020' does not match format '%Y-%m-%d' 2023-12-20 01:42:13,874 - Input arguments: {'data': ['2023-07-07', '1 June 2020', '2023.07.07', '2023-12-01', '2020/01/01', 'Nov 11 1994']} 2023-12-20 01:42:13,874 - Source Code: def perform_data_transformation(data): """takes in list of datestrings, transforms into datetime objects. """ date_format = '%Y-%m-%d' for idx, date in enumerate(data): data[idx] = datetime.strptime(date, date_format) return data 2023-12-20 01:42:13,874 - Requesting INITIAL correction - Attempt 1 2023-12-20 01:42:13,874 - API REQUEST to google 2023-12-20 01:42:13,874 - API REQUEST to gemini-pro - Temperature: 0.1 - Max Tokens: 1024 - candidate_count: 1 - Stop: None 2023-12-20 01:42:17,296 - Received INITIAL RAW suggestion: ||| import datetime def perform_data_transformation(data): """takes in list of datestrings, transforms into datetime objects. """ date_formats = ['%Y-%m-%d', '%d %B %Y', '%Y.%m.%d', '%Y-%m-%d %H:%M:%S', '%Y/%m/%d', '%b %d %Y'] for idx, date in enumerate(data): for date_format in date_formats: try: data[idx] = datetime.strptime(date, date_format) break except ValueError: continue return data ||| 2023-12-20 01:42:17,304 - Requesting TWIN review 2023-12-20 01:42:17,304 - API REQUEST to openai 2023-12-20 01:42:17,304 - API REQUEST to gpt-3.5-turbo - Temperature: 0.1 - Max Tokens: 1024 - N: 1 - Stop: None 2023-12-20 01:42:20,694 - TWIN review complete: ||| import datetime def perform_data_transformation(data): """takes in list of datestrings, transforms into datetime objects. """ date_formats = ['%Y-%m-%d', '%d %B %Y', '%Y.%m.%d', '%Y-%m-%d %H:%M:%S', '%Y/%m/%d', '%b %d %Y'] for idx, date in enumerate(data): for date_format in date_formats: try: data[idx] = datetime.datetime.strptime(date, date_format) break except ValueError: continue return data ||| 2023-12-20 01:42:20,694 - Twin Safeguard: Function name changed to ||| import datetime def perform_data_transformation(data): """takes in list of datestrings, transforms into datetime objects. """ date_formats = ['%Y-%m-%d', '%d %B %Y', '%Y.%m.%d', '%Y-%m-%d %H:%M:%S', '%Y/%m/%d', '%b %d %Y'] for idx, date in enumerate(data): for date_format in date_formats: try: data[idx] = datetime.datetime.strptime(date, date_format) break except ValueError: continue return data ||| 2023-12-20 01:42:20,694 - Received INITIAL CLEANED suggestion: import datetime def perform_data_transformation(data): """takes in list of datestrings, transforms into datetime objects. """ date_formats = ['%Y-%m-%d', '%d %B %Y', '%Y.%m.%d', '%Y-%m-%d %H:%M:%S', '%Y/%m/%d', '%b %d %Y'] for idx, date in enumerate(data): for date_format in date_formats: try: data[idx] = datetime.datetime.strptime(date, date_format) break except ValueError: continue return data 2023-12-20 01:42:20,694 - Import block added to suggested code: import datetime def perform_data_transformation(data): import datetime """takes in list of datestrings, transforms into datetime objects. """ date_formats = ['%Y-%m-%d', '%d %B %Y', '%Y.%m.%d', '%Y-%m-%d %H:%M:%S', '%Y/%m/%d', '%b %d %Y'] for idx, date in enumerate(data): for date_format in date_formats: try: data[idx] = datetime.datetime.strptime(date, date_format) break except ValueError: continue return data 2023-12-20 01:42:20,698 - Attempt 1 to reanimate 2023-12-20 01:42:20,698 - Reanimation successful, using: import datetime def perform_data_transformation(data): import datetime """takes in list of datestrings, transforms into datetime objects. """ date_formats = ['%Y-%m-%d', '%d %B %Y', '%Y.%m.%d', '%Y-%m-%d %H:%M:%S', '%Y/%m/%d', '%b %d %Y'] for idx, date in enumerate(data): for date_format in date_formats: try: data[idx] = datetime.datetime.strptime(date, date_format) break except ValueError: continue return data 2023-12-20 01:42:20,698 - Requesting human review The following is the result of the reanimation attempt: [datetime.datetime(2023, 7, 7, 0, 0), datetime.datetime(2020, 6, 1, 0, 0), datetime.datetime(2023, 7, 7, 0, 0), datetime.datetime(2023, 12, 1, 0, 0), datetime.datetime(2020, 1, 1, 0, 0), datetime.datetime(1994, 11, 11, 0, 0)] Proceed? [y/n]y [datetime.datetime(2023, 7, 7, 0, 0), datetime.datetime(2020, 6, 1, 0, 0), datetime.datetime(2023, 7, 7, 0, 0), datetime.datetime(2023, 12, 1, 0, 0), datetime.datetime(2020, 1, 1, 0, 0), datetime.datetime(1994, 11, 11, 0, 0)] ```

fukkatsu 0.0.14 - `All In`

Expand

fukkatsu 0.0.14 now supports multiple llm providers by authenticating with OpenAI. You only need to provide the `OPENAI_API_KEY`. To improve performance, the short term memory implementation now uses sqlite. Set `primary_model_api` or `secondary_model_api` to `gateway`.You will now have access to `base_url` within `primary_config` & `secondary_config` which allows you to change your llm backend.Currently `base_url` is set to `https://openrouter.ai/api/v1`. ### `resurrect example` ```python def resurrect( lives: int = 1, additional_req: str = "", allow_installs: bool = False, active_twin: bool = True, primary_model_api: str = "gateway", secondary_model_api: str = "gateway", primary_config = {"temperature": 0.01, "model": "meta-llama/llama-3.1-8b-instruct:free"}, secondary_config = {"temperature": 0.45, "model": "meta-llama/llama-3.1-8b-instruct:free"}, human_action: bool = True, active_memory: bool = True, ): ... ```

Samples - `Synthetic` Code in Action

Expand

### `resurrect` - Twin not active ```python file_path = "status_field.xlsx" @resurrect(lives=3, additional_req = "make sure that the function returns a DataFrame", allow_installs = True, active_twin = False) def read_file(file_path: str): """read file and return a data frame""" df = pd.read_csv(file_path) return df read_file(file_path) ``` #### logs

Show Full Logs

``` 2023-06-22 00:16:37,701 - 'utf-8' codec can't decode bytes in position 15-16: invalid continuation byte Traceback (most recent call last): File "c:\users\max\documents\research\fukkatsu\fukkatsu\fukkatsu\__init__.py", line 34, in wrapper result = func(*args_copy, **kwargs_copy) File "C:\Users\Max\AppData\Local\Temp\ipykernel_9256\8051789.py", line 8, in read_file df = pd.read_csv(file_path) File "C:\Users\Max\anaconda3\lib\site-packages\pandas\io\parsers\readers.py", line 912, in read_csv return _read(filepath_or_buffer, kwds) File "C:\Users\Max\anaconda3\lib\site-packages\pandas\io\parsers\readers.py", line 577, in _read parser = TextFileReader(filepath_or_buffer, **kwds) File "C:\Users\Max\anaconda3\lib\site-packages\pandas\io\parsers\readers.py", line 1407, in __init__ self._engine = self._make_engine(f, self.engine) File "C:\Users\Max\anaconda3\lib\site-packages\pandas\io\parsers\readers.py", line 1679, in _make_engine return mapping[engine](f, **self.options) File "C:\Users\Max\anaconda3\lib\site-packages\pandas\io\parsers\c_parser_wrapper.py", line 93, in __init__ self._reader = parsers.TextReader(src, **kwds) File "pandas\_libs\parsers.pyx", line 548, in pandas._libs.parsers.TextReader.__cinit__ File "pandas\_libs\parsers.pyx", line 637, in pandas._libs.parsers.TextReader._get_header File "pandas\_libs\parsers.pyx", line 848, in pandas._libs.parsers.TextReader._tokenize_rows File "pandas\_libs\parsers.pyx", line 859, in pandas._libs.parsers.TextReader._check_tokenize_status File "pandas\_libs\parsers.pyx", line 2017, in pandas._libs.parsers.raise_parser_error UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 15-16: invalid continuation byte 2023-06-22 00:16:37,705 - Input arguments: {'file_path': 'status_field.xlsx'} 2023-06-22 00:16:37,705 - Source Code: def read_file(file_path: str): """read file and return a data frame""" df = pd.read_csv(file_path) return df 2023-06-22 00:16:37,706 - Requesting INITIAL correction - Attempt 1 2023-06-22 00:16:37,707 - API REQUEST to gpt-3.5-turbo 2023-06-22 00:16:42,114 - Received INITIAL RAW suggestion: ||| import pandas as pd def read_file(file_path: str) -> pd.DataFrame: """ Read a CSV file and return a pandas DataFrame. Args: file_path (str): The path to the CSV file. Returns: pd.DataFrame: A pandas DataFrame containing the data from the CSV file. """ df = pd.read_csv(file_path, encoding='utf-8') return df ||| 2023-06-22 00:16:42,114 - Received INITIAL CLEANED suggestion: import pandas as pd def read_file(file_path: str) -> pd.DataFrame: """ Read a CSV file and return a pandas DataFrame. Args: file_path (str): The path to the CSV file. Returns: pd.DataFrame: A pandas DataFrame containing the data from the CSV file. """ df = pd.read_csv(file_path, encoding='utf-8') return df 2023-06-22 00:16:42,114 - Import block added to suggested code: import pandas as pd def read_file(file_path: str) -> pd.DataFrame: import pandas as pd """ Read a CSV file and return a pandas DataFrame. Args: file_path (str): The path to the CSV file. Returns: pd.DataFrame: A pandas DataFrame containing the data from the CSV file. """ df = pd.read_csv(file_path, encoding='utf-8') return df 2023-06-22 00:16:42,114 - Attempt 1 to reanimate 2023-06-22 00:16:42,120 - 'utf-8' codec can't decode bytes in position 0-1: invalid continuation byte Traceback (most recent call last): File "c:\users\max\documents\research\fukkatsu\fukkatsu\fukkatsu\__init__.py", line 34, in wrapper result = func(*args_copy, **kwargs_copy) File "C:\Users\Max\AppData\Local\Temp\ipykernel_9256\8051789.py", line 8, in read_file df = pd.read_csv(file_path) File "C:\Users\Max\anaconda3\lib\site-packages\pandas\io\parsers\readers.py", line 912, in read_csv return _read(filepath_or_buffer, kwds) File "C:\Users\Max\anaconda3\lib\site-packages\pandas\io\parsers\readers.py", line 577, in _read parser = TextFileReader(filepath_or_buffer, **kwds) File "C:\Users\Max\anaconda3\lib\site-packages\pandas\io\parsers\readers.py", line 1407, in __init__ self._engine = self._make_engine(f, self.engine) File "C:\Users\Max\anaconda3\lib\site-packages\pandas\io\parsers\readers.py", line 1679, in _make_engine return mapping[engine](f, **self.options) File "C:\Users\Max\anaconda3\lib\site-packages\pandas\io\parsers\c_parser_wrapper.py", line 93, in __init__ self._reader = parsers.TextReader(src, **kwds) File "pandas\_libs\parsers.pyx", line 548, in pandas._libs.parsers.TextReader.__cinit__ File "pandas\_libs\parsers.pyx", line 637, in pandas._libs.parsers.TextReader._get_header File "pandas\_libs\parsers.pyx", line 848, in pandas._libs.parsers.TextReader._tokenize_rows File "pandas\_libs\parsers.pyx", line 859, in pandas._libs.parsers.TextReader._check_tokenize_status File "pandas\_libs\parsers.pyx", line 2017, in pandas._libs.parsers.raise_parser_error UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 15-16: invalid continuation byte During handling of the above exception, another exception occurred: Traceback (most recent call last): File "c:\users\max\documents\research\fukkatsu\fukkatsu\fukkatsu\__init__.py", line 116, in wrapper output = new_function(*args_copy, **kwargs_copy) File "", line 14, in read_file File "C:\Users\Max\anaconda3\lib\site-packages\pandas\io\parsers\readers.py", line 912, in read_csv return _read(filepath_or_buffer, kwds) File "C:\Users\Max\anaconda3\lib\site-packages\pandas\io\parsers\readers.py", line 577, in _read parser = TextFileReader(filepath_or_buffer, **kwds) File "C:\Users\Max\anaconda3\lib\site-packages\pandas\io\parsers\readers.py", line 1407, in __init__ self._engine = self._make_engine(f, self.engine) File "C:\Users\Max\anaconda3\lib\site-packages\pandas\io\parsers\readers.py", line 1679, in _make_engine return mapping[engine](f, **self.options) File "C:\Users\Max\anaconda3\lib\site-packages\pandas\io\parsers\c_parser_wrapper.py", line 93, in __init__ self._reader = parsers.TextReader(src, **kwds) File "pandas\_libs\parsers.pyx", line 548, in pandas._libs.parsers.TextReader.__cinit__ File "pandas\_libs\parsers.pyx", line 665, in pandas._libs.parsers.TextReader._get_header UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 0-1: invalid continuation byte 2023-06-22 00:16:42,124 - Reanimation failed, requesting new correction 2023-06-22 00:16:42,124 - API REQUEST to gpt-3.5-turbo 2023-06-22 00:16:45,294 - Received attempt RAW suggestion: ||| import pandas as pd def read_file(file_path: str) -> pd.DataFrame: try: df = pd.read_csv(file_path, encoding='utf-8') except UnicodeDecodeError: df = pd.read_excel(file_path) return df ||| 2023-06-22 00:16:45,294 - Received attempt CLEANED suggestion: import pandas as pd def read_file(file_path: str) -> pd.DataFrame: try: df = pd.read_csv(file_path, encoding='utf-8') except UnicodeDecodeError: df = pd.read_excel(file_path) return df 2023-06-22 00:16:45,294 - Import block added to suggested code: import pandas as pd def read_file(file_path: str) -> pd.DataFrame: import pandas as pd try: df = pd.read_csv(file_path, encoding='utf-8') except UnicodeDecodeError: df = pd.read_excel(file_path) return df 2023-06-22 00:16:45,294 - Attempt 2 to reanimate 2023-06-22 00:16:45,308 - Reanimation successful, using: import pandas as pd def read_file(file_path: str) -> pd.DataFrame: import pandas as pd try: df = pd.read_csv(file_path, encoding='utf-8') except UnicodeDecodeError: df = pd.read_excel(file_path) return df ```

Output

ID Field Cost Country Status 0 1 Eng 200000 Germany active 1 1 Eng 200000 Italy active 2 1 Eng 200000 UK active 3 1 Eng 400500 US active 4 1 Eng 100500 Italy active 5 1 Eng 100500 Italy deactivated 6 1 Eng 100500 Spain active

`resurrect` - Twin active

```python filepath = "statusfield.xlsx"

@resurrect(lives=3, additionalreq = "make sure that the function returns a DataFrame", allowinstalls = True, activetwin = True) def readfile(filepath: str): """read file and return a data frame""" df = pd.readcsv(file_path) return df

readfile(filepath) ```

logs

Show Full Logs

``` 2023-06-22 00:19:40,599 - 'utf-8' codec can't decode bytes in position 15-16: invalid continuation byte Traceback (most recent call last): File "c:\users\max\documents\research\fukkatsu\fukkatsu\fukkatsu\__init__.py", line 34, in wrapper result = func(*args_copy, **kwargs_copy) File "C:\Users\Max\AppData\Local\Temp\ipykernel_9256\423974772.py", line 8, in read_file df = pd.read_csv(file_path) File "C:\Users\Max\anaconda3\lib\site-packages\pandas\io\parsers\readers.py", line 912, in read_csv return _read(filepath_or_buffer, kwds) File "C:\Users\Max\anaconda3\lib\site-packages\pandas\io\parsers\readers.py", line 577, in _read parser = TextFileReader(filepath_or_buffer, **kwds) File "C:\Users\Max\anaconda3\lib\site-packages\pandas\io\parsers\readers.py", line 1407, in __init__ self._engine = self._make_engine(f, self.engine) File "C:\Users\Max\anaconda3\lib\site-packages\pandas\io\parsers\readers.py", line 1679, in _make_engine return mapping[engine](f, **self.options) File "C:\Users\Max\anaconda3\lib\site-packages\pandas\io\parsers\c_parser_wrapper.py", line 93, in __init__ self._reader = parsers.TextReader(src, **kwds) File "pandas\_libs\parsers.pyx", line 548, in pandas._libs.parsers.TextReader.__cinit__ File "pandas\_libs\parsers.pyx", line 637, in pandas._libs.parsers.TextReader._get_header File "pandas\_libs\parsers.pyx", line 848, in pandas._libs.parsers.TextReader._tokenize_rows File "pandas\_libs\parsers.pyx", line 859, in pandas._libs.parsers.TextReader._check_tokenize_status File "pandas\_libs\parsers.pyx", line 2017, in pandas._libs.parsers.raise_parser_error UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 15-16: invalid continuation byte 2023-06-22 00:19:40,604 - Input arguments: {'file_path': 'status_field.xlsx'} 2023-06-22 00:19:40,605 - Source Code: def read_file(file_path: str): """read file and return a data frame""" df = pd.read_csv(file_path) return df 2023-06-22 00:19:40,606 - Requesting INITIAL correction - Attempt 1 2023-06-22 00:19:40,607 - API REQUEST to gpt-3.5-turbo 2023-06-22 00:19:44,843 - Received INITIAL RAW suggestion: ||| import pandas as pd def read_file(file_path: str) -> pd.DataFrame: """Reads a CSV file and returns a pandas DataFrame. Args: file_path (str): The path to the CSV file. Returns: pd.DataFrame: The pandas DataFrame containing the data from the CSV file. """ df = pd.read_csv(file_path, encoding='utf-8') return df ||| 2023-06-22 00:19:44,843 - Requesting TWIN review 2023-06-22 00:19:44,843 - API REQUEST to gpt-3.5-turbo 2023-06-22 00:19:50,260 - TWIN review complete: ||| import pandas as pd def read_file(file_path: str, sheet_name: str = None) -> pd.DataFrame: """ Reads an Excel file and returns a pandas DataFrame. Args: file_path (str): The path to the Excel file. sheet_name (str, optional): The name of the sheet to read. Defaults to None. Returns: pd.DataFrame: The pandas DataFrame containing the data from the Excel file. """ df = pd.read_excel(file_path, sheet_name=sheet_name) return df ||| 2023-06-22 00:19:50,260 - Twin Safeguard: Function name changed to ||| import pandas as pd def read_file(file_path: str, sheet_name: str = None) -> pd.DataFrame: """ Reads an Excel file and returns a pandas DataFrame. Args: file_path (str): The path to the Excel file. sheet_name (str, optional): The name of the sheet to read. Defaults to None. Returns: pd.DataFrame: The pandas DataFrame containing the data from the Excel file. """ df = pd.read_excel(file_path, sheet_name=sheet_name) return df ||| 2023-06-22 00:19:50,260 - Received INITIAL CLEANED suggestion: import pandas as pd def read_file(file_path: str, sheet_name: str = None) -> pd.DataFrame: """ Reads an Excel file and returns a pandas DataFrame. Args: file_path (str): The path to the Excel file. sheet_name (str, optional): The name of the sheet to read. Defaults to None. Returns: pd.DataFrame: The pandas DataFrame containing the data from the Excel file. """ df = pd.read_excel(file_path, sheet_name=sheet_name) return df 2023-06-22 00:19:50,260 - Import block added to suggested code: import pandas as pd def read_file(file_path: str, sheet_name: str = None) -> pd.DataFrame: import pandas as pd """ Reads an Excel file and returns a pandas DataFrame. Args: file_path (str): The path to the Excel file. sheet_name (str, optional): The name of the sheet to read. Defaults to None. Returns: pd.DataFrame: The pandas DataFrame containing the data from the Excel file. """ df = pd.read_excel(file_path, sheet_name=sheet_name) return df 2023-06-22 00:19:50,260 - Attempt 1 to reanimate 2023-06-22 00:19:50,275 - Reanimation successful, using: import pandas as pd def read_file(file_path: str, sheet_name: str = None) -> pd.DataFrame: import pandas as pd """ Reads an Excel file and returns a pandas DataFrame. Args: file_path (str): The path to the Excel file. sheet_name (str, optional): The name of the sheet to read. Defaults to None. Returns: pd.DataFrame: The pandas DataFrame containing the data from the Excel file. """ df = pd.read_excel(file_path, sheet_name=sheet_name) return df ```

Output

{'Sheet1': ID Field Cost Country Status 0 1 Eng 200000 Germany active 1 1 Eng 200000 Italy active 2 1 Eng 200000 UK active 3 1 Eng 400500 US active 4 1 Eng 100500 Italy active 5 1 Eng 100500 Italy deactivated 6 1 Eng 100500 Spain active}

`mutate` - Twin not active

```python filepath = "statusfield.xlsx"

@mutate(request="look at the input file, make sure to change the function according to the file.") def readfile(filepath: str): """read file and return a data frame""" df = pd.readcsv(filepath) return df

readfile(filepath) ```

logs

Show Full Logs

``` 2023-06-22 00:30:25,589 - Input arguments: {'file_path': 'status_field.xlsx'} 2023-06-22 00:30:25,590 - Source Code: def read_file(file_path: str): """read file and return a data frame""" df = pd.read_csv(file_path) return df 2023-06-22 00:30:25,592 - Requesting mutation 2023-06-22 00:30:25,592 - API REQUEST to gpt-3.5-turbo 2023-06-22 00:30:31,373 - Received RAW suggestion mutation: ||| import pandas as pd def read_file(file_path: str): """ Read file and return a data frame. Args: file_path (str): The path of the file to be read. Returns: pandas.DataFrame: The data frame containing the data from the file. """ if file_path.endswith('.csv'): df = pd.read_csv(file_path) elif file_path.endswith('.xlsx'): df = pd.read_excel(file_path) else: raise ValueError('File format not supported. Please provide a CSV or Excel file.') return df ||| 2023-06-22 00:30:31,373 - Received CLEANED suggestion mutation: import pandas as pd def read_file(file_path: str): """ Read file and return a data frame. Args: file_path (str): The path of the file to be read. Returns: pandas.DataFrame: The data frame containing the data from the file. """ if file_path.endswith('.csv'): df = pd.read_csv(file_path) elif file_path.endswith('.xlsx'): df = pd.read_excel(file_path) else: raise ValueError('File format not supported. Please provide a CSV or Excel file.') return df 2023-06-22 00:30:31,373 - Import block added to suggested code: import pandas as pd def read_file(file_path: str): import pandas as pd """ Read file and return a data frame. Args: file_path (str): The path of the file to be read. Returns: pandas.DataFrame: The data frame containing the data from the file. """ if file_path.endswith('.csv'): df = pd.read_csv(file_path) elif file_path.endswith('.xlsx'): df = pd.read_excel(file_path) else: raise ValueError('File format not supported. Please provide a CSV or Excel file.') return df 2023-06-22 00:30:31,386 - Mutation successful, using import pandas as pd def read_file(file_path: str): import pandas as pd """ Read file and return a data frame. Args: file_path (str): The path of the file to be read. Returns: pandas.DataFrame: The data frame containing the data from the file. """ if file_path.endswith('.csv'): df = pd.read_csv(file_path) elif file_path.endswith('.xlsx'): df = pd.read_excel(file_path) else: raise ValueError('File format not supported. Please provide a CSV or Excel file.') return df ```

Output

ID Field Cost Country Status 0 1 Eng 200000 Germany active 1 1 Eng 200000 Italy active 2 1 Eng 200000 UK active 3 1 Eng 400500 US active 4 1 Eng 100500 Italy active 5 1 Eng 100500 Italy deactivated 6 1 Eng 100500 Spain active

`stalk` - Twin not active

```python @stalk(likelihood = 0.6, additionalreq = "", allowinstalls = False, activetwin = False, llm = {"primary": "gpt-3.5-turbo", "secondary": "gpt-3.5-turbo"}, temperature = {"primary": 0.1, "secondary": 0.1}) def myfunction(x, y, z): """ function to divide x by y and add to the result z. Should return z if y is 0. """ result = x / y + z return result

print(my_function(x = 1, y = 0, z= 2)) ```

logs

Show Full Logs

``` 2023-06-22 00:39:25,914 - Random number: 0.2695059864882857, Likelihood: 0.6 2023-06-22 00:39:25,916 - Input arguments: {'x': 1, 'y': 0, 'z': 2} 2023-06-22 00:39:25,918 - Source Code: def my_function(x, y, z): """ function to divide x by y and add to the result z. Should return z if y is 0. """ result = x / y + z return result 2023-06-22 00:39:25,919 - Stalking function 2023-06-22 00:39:25,920 - API REQUEST to gpt-3.5-turbo 2023-06-22 00:39:30,115 - Received RAW suggestion from Stalker: ||| def my_function(x, y, z): """ This function divides x by y and adds to the result z. If y is 0, it returns z. Time complexity: O(1) Space complexity: O(1) """ if y == 0: return z result = x / y + z return result ||| 2023-06-22 00:39:30,115 - Received CLEANED suggestion review: def my_function(x, y, z): """ This function divides x by y and adds to the result z. If y is 0, it returns z. Time complexity: O(1) Space complexity: O(1) """ if y == 0: return z result = x / y + z return result 2023-06-22 00:39:30,115 - Import block added to suggested code: def my_function(x, y, z): """ This function divides x by y and adds to the result z. If y is 0, it returns z. Time complexity: O(1) Space complexity: O(1) """ if y == 0: return z result = x / y + z return result 2023-06-22 00:39:30,115 - Review successful, using def my_function(x, y, z): """ This function divides x by y and adds to the result z. If y is 0, it returns z. Time complexity: O(1) Space complexity: O(1) """ if y == 0: return z result = x / y + z return result ```

Output

2

Testing and measuring fukkatsu's Capabilities

The following section delves into a series of simulations aimed at gaining a deeper understanding of fukkatsu's potential capabilities.

Please follow this Link for more information on fukkatsu's performance.

Legacy MVP

Expand

You can find a MVP within the `poc` folder. You can simply run the code via `python mvp.py`. The code will simulate a failing function, which will be repaird during execution. The mvp.py code will not request a correction to an OpenAi LLM but simply ueses a mock corrected function. ### Foundation #### Example: - we have a function called `my_function` which takes accepts three arguments: 'x', 'y', 'z' and returns a value calculated via `x / y + z` - lets assume the function `my_function` accidentally receives the value 0 for the argument 'y' - this will cause the function to fail with a `ZeroDivisionError` becaue it was not accounted for in the original function - fukkatsu offers a second chance here via the @mvp_reanimate decorator - the decorator will catch the error and request a correction from an OpenAi LLM such as `gpt-3.5-turbo`. - the corrected function will recieve the orignal arguments and handle the error as intended - to get the most of the correction ability of fukkatsu, it will be paramount for the user to provide a good description of the function and its intended purpose via a well defined docstring - fukkatsu makes sure that the LLM will receive all the necessary information to correct the function without changing its original purpose: - Full error traceback - original function code - passed arguments ```python @mvp_reanimate def my_function(x, y, z): """ function to divide x by y and add to the result z. Should return z if y is 0. """ result = x / y + z return result print(my_function(x = 1, y = 0, z= 2)) # would fail, but is corrected and returns 2 print(my_function(x = 2, y = 0, z= 10)) # would fail, but is corrected and returns 10 print(my_function(x = 9, y = 1, z= 2) + 10 ) # would not fail, returns 21.0 ``` Please note, the example in the above is trivial however LLMs such as `gpt-3.5-turbo` are able to correct more complex functions. Once the library is more mature, more experiments and examples will show if such a use case for LLMs is worthwhile. ### Extra life Here is again a representation of what I am trying to achieve: https://media.tenor.com/r5nBe8Ft6yEAAAAC/ready-player-one-extra-life.gif The code mvp code offers now the concept of `extra lives`. The idea of extra lives is to allow the user to define, per function, how often a LLM should attempt to fix errors. This will allow LLMs to futher explore other paths of fixing the code at runtime however it will also make sure to bound the runtime of the LLM. #### Example: ```python @mvp_reanimate(lives=2) def my_function(x, y, z): """ function to divide x by y and add to the result z. Should return z if y is 0. """ result = x / y + z return result ``` The above example will allow the LLM to attempt to fix the function twice. If the LLM fails to fix the function after two attempts, a `flatline error` will be raised which indicates that the LLM was not able to fix the function during runtime.

Owner

Name: Max Mekiska
Login: maxmekiska
Kind: user
Location: London

Repositories: 2
Profile: https://github.com/maxmekiska

Citation (CITATION.cff)

cff-version: 1.2.0
title: fukkatsu
message: >-
  If you use this software, please cite it using the
  metadata from this file.
type: software
authors:
  - given-names: Maximilian Alexander
    family-names: Mekiska
    email: maxmekiska@gmail.com
repository-code: 'https://github.com/maxmekiska/fukkatsu/tree/main'
abstract: >-
  Dynamic Software Improvement and Mutation using LLMs for
  Stochastic Synthetic Code Injections.
keywords:
  - LLM
  - ML
license: MIT
commit: d9dc4e11fdd73fff740bc49dae09e712ad97e59f
version: 0.0.14
date-released: '2023-12-08'

GitHub Events

Total

Release event: 1
Push event: 6
Create event: 1

Last Year

Release event: 1
Push event: 6
Create event: 1

Committers

Last synced: over 1 year ago

All Time

Total Commits: 90
Total Committers: 1
Avg Commits per committer: 90.0
Development Distribution Score (DDS): 0.0

Past Year

Commits: 13
Committers: 1
Avg Commits per committer: 13.0
Development Distribution Score (DDS): 0.0

Top Committers

Name	Email	Commits
maxmekiska	m**a@g**m	90

Issues and Pull Requests

Last synced: 7 months ago

All Time

Total issues: 0
Total pull requests: 0
Average time to close issues: N/A
Average time to close pull requests: N/A
Total issue authors: 0
Total pull request authors: 0
Average comments per issue: 0
Average comments per pull request: 0
Merged pull requests: 0
Bot issues: 0
Bot pull requests: 0

Past Year

Issues: 0
Pull requests: 0
Average time to close issues: N/A
Average time to close pull requests: N/A
Issue authors: 0
Pull request authors: 0
Average comments per issue: 0
Average comments per pull request: 0
Merged pull requests: 0
Bot issues: 0
Bot pull requests: 0

View more stats

Top Authors

Issue Authors

Pull Request Authors

Top Labels

Issue Labels

Pull Request Labels

Packages

Total packages: 1
Total downloads:
- pypi 49 last-month

Total dependent packages: 0
Total dependent repositories: 0
Total versions: 14
Total maintainers: 1

pypi.org: fukkatsu

Dynamic Software Improvement and Mutation using LLMs for Stochastic Synthetic Code Injections.

Homepage: https://github.com/maxmekiska/fukkatsu
Documentation: https://fukkatsu.readthedocs.io/
License: mit
Latest release: 0.0.14
published over 1 year ago

Versions: 14
Dependent Packages: 0
Dependent Repositories: 0
Downloads: 49 Last month

Rankings

Dependent packages count: 7.3%

Average: 27.8%

Forks count: 30.5%

Stargazers count: 32.5%

Dependent repos count: 40.9%

Maintainers (1)

mmekiska

Last synced: 7 months ago

fukkatsu

Science Score: 44.0%

Keywords

Repository

Basic Info

Statistics

Topics

Metadata Files

README.md

fukkatsu

API Keys

Description

Medium post

Quick Start

fukkatsu 0.0.1 - Extra Life

fukkatsu 0.0.2 - The Ghost in the Machine

fukkatsu 0.0.3 - Laissez-faire

fukkatsu 0.0.5 - Not so Evil Twin

fukkatsu 0.0.8 - I can see you

fukkatsu 0.0.10 - Sharing is Caring

fukkatsu 0.0.11 - The Humans are back

fukkatsu 0.0.13 - Making new Friends

fukkatsu 0.0.14 - All In

Samples - Synthetic Code in Action

Output

resurrect - Twin active

logs

Output

mutate - Twin not active

logs

Output

stalk - Twin not active

logs

Output

Testing and measuring fukkatsu's Capabilities

Legacy MVP

Owner

Citation (CITATION.cff)

GitHub Events

Total

Last Year

Committers

All Time

Past Year

Top Committers

Issues and Pull Requests

All Time

Past Year

Top Authors

Issue Authors

Pull Request Authors

Top Labels

Issue Labels

Pull Request Labels

Packages

pypi.org: fukkatsu

Rankings

Maintainers (1)

fukkatsu 0.0.1 - `Extra Life`

fukkatsu 0.0.2 - `The Ghost in the Machine`

fukkatsu 0.0.3 - `Laissez-faire`

fukkatsu 0.0.5 - `Not so Evil Twin`

fukkatsu 0.0.8 - `I can see you`

fukkatsu 0.0.10 - `Sharing is Caring`

fukkatsu 0.0.11 - `The Humans are back`

fukkatsu 0.0.13 - `Making new Friends`

fukkatsu 0.0.14 - `All In`

Samples - `Synthetic` Code in Action

`resurrect` - Twin active

`mutate` - Twin not active

`stalk` - Twin not active