Skip to content

Add YamlPackSerializer #967

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,7 @@ How does it work
Aiocache provides 3 main entities:

- **backends**: Allow you specify which backend you want to use for your cache. Currently supporting: SimpleMemoryCache, RedisCache using redis_ and MemCache using aiomcache_.
- **serializers**: Serialize and deserialize the data between your code and the backends. This allows you to save any Python object into your cache. Currently supporting: StringSerializer, PickleSerializer, JsonSerializer, and MsgPackSerializer. But you can also build custom ones.
- **serializers**: Serialize and deserialize the data between your code and the backends. This allows you to save any Python object into your cache. Currently supporting: StringSerializer, PickleSerializer, JsonSerializer, MsgPackSerializer, and YamlSerializer. But you can also build custom ones.
- **plugins**: Implement a hooks system that allows to execute extra behavior before and after of each command.

If you are missing an implementation of backend, serializer or plugin you think it could be interesting for the package, do not hesitate to open a new issue.
Expand Down
2 changes: 2 additions & 0 deletions aiocache/serializers/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
NullSerializer,
PickleSerializer,
StringSerializer,
YamlSerializer,
)

logger = logging.getLogger(__name__)
Expand All @@ -28,4 +29,5 @@
"PickleSerializer",
"JsonSerializer",
"MsgPackSerializer",
"YamlSerializer",
]
38 changes: 38 additions & 0 deletions aiocache/serializers/serializers.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,12 @@
msgpack = None
logger.debug("msgpack not installed, MsgPackSerializer unavailable")

try:
import yaml # noqa: I900
except ImportError:
yaml = None
logger.debug("yaml not installed, YamlSerializer unavailable")


_NOT_SET = object()

Expand Down Expand Up @@ -197,3 +203,35 @@ def loads(self, value):
if value is None:
return None
return msgpack.loads(value, raw=raw, use_list=self.use_list)


class YamlSerializer(BaseSerializer):
"""
Transform data to YAML string with ``yaml.dump`` and ``yaml.load`` to retrieve it back. You need
to have ``yaml`` installed in order to be able to use this serializer.
"""

def __init__(self, *args, **kwargs):
if not yaml:
raise RuntimeError("yaml not installed, YamlSerializer unavailable")
super().__init__(*args, **kwargs)

def dumps(self, value):
"""
Serialize the received value using ``yaml.dump``.

:param value: obj
:returns: str
"""
return yaml.dump(value)

def loads(self, value):
"""
Deserialize value using ``yaml.load``.

:param value: str
:returns: obj
"""
if value is None:
return None
return yaml.safe_load(value)
10 changes: 9 additions & 1 deletion docs/serializers.rst
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,14 @@ MsgPackSerializer
.. autoclass:: aiocache.serializers.MsgPackSerializer
:members:

.. _yamlserializer:

YamlSerializer
--------------

.. autoclass:: aiocache.serializers.YamlSerializer
:members:

In case the current serializers are not covering your needs, you can always define your custom serializer as shown in ``examples/serializer_class.py``:

.. literalinclude:: ../examples/serializer_class.py
Expand All @@ -66,4 +74,4 @@ You can also use marshmallow as your serializer (``examples/marshmallow_serializ
:language: python
:linenos:

By default cache backends assume they are working with ``str`` types. If your custom implementation transform data to bytes, you will need to set the class attribute ``encoding`` to ``None``.
By default cache backends assume they are working with ``str`` types. If your custom implementation transform data to bytes, you will need to set the class attribute ``encoding`` to ``None``.
1 change: 1 addition & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,4 @@ pytest-asyncio==0.25.3
pytest-cov==6.0.0
pytest-mock==3.14.0
redis==5.2.1
pyyaml==6.0.2
42 changes: 42 additions & 0 deletions tests/ut/test_serializers.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,15 @@
NullSerializer,
PickleSerializer,
StringSerializer,
YamlSerializer,
)


Dummy = namedtuple("Dummy", "a, b")

TYPES = [1, 2.0, "hi", True, ["1", 1], {"key": "value"}, Dummy(1, 2)]
JSON_TYPES = [1, 2.0, "hi", True, ["1", 1], {"key": "value"}]
YAML_TYPES = [1, 2.0, "hi", True, ["1", 1], {"key": "value"}]


class TestNullSerializer:
Expand Down Expand Up @@ -173,3 +175,43 @@ def test_dumps_and_loads_dict(self):
"a": [1, 2, ["1", 2]],
"b": {"b": 1, "c": [1, 2]},
}


class TestYamlSerializer:
def test_init(self):
serializer = YamlSerializer()
assert isinstance(serializer, BaseSerializer)
assert serializer.DEFAULT_ENCODING == "utf-8"
assert serializer.encoding == "utf-8"

def test_init_fails_if_yaml_not_installed(self):
with mock.patch("aiocache.serializers.serializers.yaml", None):
with pytest.raises(RuntimeError):
YamlSerializer()
assert JsonSerializer(), "Other serializers should still initialize"

@pytest.mark.parametrize("obj", YAML_TYPES)
def test_set_types(self, obj):
serializer = YamlSerializer()
assert serializer.loads(serializer.dumps(obj)) == obj

def test_dumps(self):
serializer = YamlSerializer()
assert serializer.dumps({"hi": 1}) == "hi: 1\n"

def test_dumps_with_none(self):
serializer = YamlSerializer()
assert serializer.dumps(None) == "null\n...\n"

def test_loads(self):
serializer = YamlSerializer()
assert serializer.loads("hi: 1\n") == {"hi": 1}

def test_loads_with_none(self):
serializer = YamlSerializer()
assert serializer.loads(None) is None

def test_dumps_and_loads(self):
obj = {"hi": 1}
serializer = YamlSerializer()
assert serializer.loads(serializer.dumps(obj)) == obj