Skip to content

Added deactivated usage. Fixed classes decoration. Ready for v0.1.10 #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 18 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,14 @@ pip install tempit

## Usage

Tempit decorator should only be used for benchmarking; it is not intended for production code.
`tempit` decorator should only be used for benchmarking and is not intended for production code. It is possible to deactivate the usage of `tempit` globally by setting the `TempitConfig.ACTIVE` flag to false as shown below:

```python

from tempit import TempitConfig, tempit
TempitConfig.ACTIVE = False # Deactivates the decorator

```

Below are some examples demonstrating `tempit`'s usage:

Expand Down Expand Up @@ -77,10 +84,11 @@ More examples can be found in the [examples.py](https://github.com/mcrespoae/tem

- Simplified usage.
- Accurate measurement of function execution time.
- Support for functions, methods, `classmethod`, `staticmethods` and classes.
- Parallel execution mode for performance measurement.
- Support for functions, methods, `classmethod` and `staticmethods`.
- Human-readable time formatting.
- Optional verbose mode for detailed information.
- Ability to globally deactivate the `tempit` decorator.
- Parallel execution mode for performance measurement.
- Automatic recursion checker.

## Parameters
Expand All @@ -102,7 +110,7 @@ The ideal way to use this package is by applying the decorator to the functions

- Recursive functions should be encapsulated for better benchmarking. Please refer to the [Recursive functions](#recursive-functions) section to to learn more about recursion and `tempit`.

- Avoid decorating classes directly, as a `PicklingError` may occur if the class is then instantiated in another process. For more information, please see the [Decorating classes that could be instantiated in other processes](#decorating-classes-that-could-be-instantiated-in-other-processes) in the [Other Limitations](#other-limitations) section.
- Decorating classes will return the class unmodified and will not be decorated. For more information about this decision, please see the [Why is class decoration bypassed](#why-is-class-decoration-bypassed) in the [Other Limitations](#other-limitations) section.

## Recursive functions

Expand Down Expand Up @@ -169,19 +177,21 @@ That being said, timings measured when using concurrent executions may not be as

## Other limitations

### Decorating classes that could be instantiated in other processes
While this package generally delivers excellent performance and reliability, it's essential to be aware of certain scenarios where using the `tempit` decorator could lead to unexpected behavior.

While this package generally delivers excellent performance and reliability, it's essential to be aware of certain scenarios where using the `tempit` decorator could lead to unexpected behavior or crashes:
### Why is class decoration bypassed?

- If a class is decorated with `tempit`, and subsequently, a new process is spawned after creating an instance of the class, calling a method within the newly created process may result in a `PicklingError`.
When a class is decorated using `tempit`, it remains unmodified and is not decorated. If the user intends to measure the time of `__self__` or any other constructor, it can be done directly on those methods.

This design decision was made due to a potential issue that arises when a decorated class is used in conjunction with spawning a new process. Specifically, if a class decorated with `tempit` is pickled for use in a separate process and then a method is called within that new process, it may result in a `PicklingError`.

This limitation arises due to how Python's pickling mechanism handles decorated classes and processes. When a decorated class instance is pickled for use in a separate process, inconsistencies in object references can occur, leading to pickling failures.

To mitigate this issue, avoid decorating classes that will be used in processes spawned later in the program's execution.

### Zero values

In some rare cases where multiple recursively decorated functions are called nested within each other, `tempit` may return two values, with one being zero.
In some rare cases where multiple recursively decorated functions are called nested within each other, `tempit` may return some zero values for the measurements of the inner functions.

## Error management and warnings

Expand All @@ -193,8 +203,6 @@ If an error occurs while executing the decorated function in sequential mode or

- Deprecation warnings will be added before removing a feature.

- Decorated classes will raise a warning to inform the user about the potential issues described in [Decorating classes that could be instantiated in other processes](#decorating-classes-that-could-be-instantiated-in-other-processes) section.

- If the `run_times` parameter exceeds `num_processors - 1`, it will be downsized to match the number of available processors minus one to maximize parallelization.

- If recursion has been detected, a warning will be prompted. If so, please, go to [Recursive functions](#recursive-functions).
Expand Down
92 changes: 73 additions & 19 deletions examples/examples.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,18 @@

from tempit import tempit

# Deactivate the decorator
# from tempit import TempitConfig
# TempitConfig.ACTIVE = True # Activates the decorator. Default option
# TempitConfig.ACTIVE = False # Deactivates the decorator


@tempit(run_times=1)
class TempitTestClassWithArgs:
@tempit
def tempit_basic(self, a: int = 1, b: int = 2):
return a + b


@tempit
class TempitTestClass:
Expand Down Expand Up @@ -134,7 +146,7 @@ def main():
print("---CLASS EXAMPLES---")

print("Once in class basic")
_ = test_class.tempit_basic(1, b=2)
_ = test_class.tempit_basic(2, b=4)
print("Concurrency 5 times in class")
test_class.tempit_5times()
print("Concurrency 5 times in class verbose")
Expand All @@ -144,34 +156,74 @@ def main():

print("Multithreading crash in class")
# If crash while running in a separate thread or process, the execution will be done in the main thread not concurrently
_ = test_class.tempit_5times_multithreading_crash(1, b=2)
_ = test_class.tempit_5times_multithreading_crash(2, b=4)

print("Method with args")
_ = test_class.args_method(1, b=2)
_ = test_class.args_method(2, b=4)
print("Static method")
_ = test_class.static_method(1, b=2)
_ = test_class.static_method(2, b=4)
print("Class method")
_, _ = test_class.class_method(1, b=2)
_, _ = test_class.class_method(2, b=4)

new_test_class = TempitTestClassWithArgs()
# Test with calling the function from another thread and process
with ThreadPoolExecutor(max_workers=1) as executor:
print("Other thread methods")
future_basic_method = executor.submit(test_class.tempit_basic)
future_basic_method.result()
future_class_method = executor.submit(test_class.class_method, 1, b=2)
_ = future_basic_method.result()

future_class_method = executor.submit(test_class.class_method, 2, b=4)
_, _ = future_class_method.result()

future_static_method = executor.submit(test_class.static_method, 2, b=4)
_ = future_static_method.result()

future_basic_method = executor.submit(new_test_class.tempit_basic)
_ = future_basic_method.result()

new_test_class = TempitTestClass()
future_basic_method = executor.submit(new_test_class.tempit_basic)
_ = future_basic_method.result()

future_class_method = executor.submit(new_test_class.class_method, 2, b=4)
_, _ = future_class_method.result()

future_static_method = executor.submit(new_test_class.static_method, 2, b=4)
_ = future_static_method.result()

new_test_class = TempitTestClassWithArgs()
future_basic_method = executor.submit(new_test_class.tempit_basic)
_ = future_basic_method.result()

with ProcessPoolExecutor(max_workers=1) as executor:
print("Other process methods.")
future_basic_method = executor.submit(test_class.tempit_basic)
_ = future_basic_method.result()

future_class_method = executor.submit(test_class.class_method, 2, b=4)
_, _ = future_class_method.result()

future_static_method = executor.submit(test_class.static_method, 2, b=4)
_ = future_static_method.result()

future_basic_method = executor.submit(new_test_class.tempit_basic)
_ = future_basic_method.result()

new_test_class = TempitTestClass()
future_basic_method = executor.submit(new_test_class.tempit_basic)
_ = future_basic_method.result()

future_class_method = executor.submit(new_test_class.class_method, 2, b=4)
_, _ = future_class_method.result()
future_static_method = executor.submit(test_class.static_method, 1, b=2)

future_static_method = executor.submit(new_test_class.static_method, 2, b=4)
_ = future_static_method.result()

if False: # This crashes at the moment if the class is decorated with @tempit
with ProcessPoolExecutor(max_workers=1) as executor:
print("Other process methods. This part crashes")
future_basic_method = executor.submit(test_class.tempit_basic)
future_basic_method.result()
future_class_method = executor.submit(test_class.class_method, 1, b=2)
_, _ = future_class_method.result()
future_static_method = executor.submit(test_class.static_method, 1, b=2)
_ = future_static_method.result()
new_test_class = TempitTestClassWithArgs()
future_basic_method = executor.submit(new_test_class.tempit_basic)
_ = future_basic_method.result()

# Test with calling the function from another thread and process

print("---END CLASS EXAMPLES---")

Expand All @@ -196,19 +248,21 @@ def main():
print("Other thread methods")
future_basic_func = executor.submit(tempit_basic)
future_basic_func.result()
future_args_func = executor.submit(args_func, 1, b=2)
future_args_func = executor.submit(args_func, 2, b=4)
_ = future_args_func.result()

with ProcessPoolExecutor(max_workers=1) as executor:
print("Other process methods")
future_basic_func = executor.submit(tempit_basic)
future_basic_func.result()
future_args_func = executor.submit(args_func, 1, b=2)
future_args_func = executor.submit(args_func, 2, b=4)
_ = future_args_func.result()

print("---END FUNCTION EXAMPLES---")
print("---OTHER EXAMPLES---")

_ = recursive_func(10)

_ = wrapped_recursive_func(10)
_ = tempit_with_recursive_func(10)
_ = call_long_process_sequential(16)
Expand Down
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
from setuptools import setup

VERSION = "0.1.9"
VERSION = "0.1.10"
setup(
name="tempit",
version=VERSION,
Expand Down
4 changes: 2 additions & 2 deletions tempit/__init__.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
from .core import tempit
from .core import TempitConfig, tempit

__all__ = ["tempit"]
__all__ = ["tempit", "TempitConfig"]
Loading