Merge pull request #223 from grillazz/switch-logger-to-rotoger

Switch logger to rotoger

README.md

@@ -24,7 +24,7 @@
       <ul>
         <li><a href="#make-will-help-you">Make will help you</a></li>
         <li><a href="#how-to-feed-database">How to feed database</a></li>
-        <li><a href="#rainbow-logs-with-rich">Rainbow logs with rich</a></li>
+        <li><a href="#structured-asynchronous-logging-with-rotoger">Structured & Asynchronous Logging with Rotoger</a></li>
         <li><a href="#setup-user-auth">Setup user auth</a></li>
         <li><a href="#setup-local-env-with-uv">Setup local development with uv</a></li>
         <li><a href="#import-xlsx-files-with-polars-and-calamine">Import xlsx files with polars and calamine</a></li>
@@ -102,27 +102,21 @@ Next models were generated with https://github.com/agronholm/sqlacodegen
 
 <p align="right">(<a href="#readme-top">back to top</a>)</p>
 
-### Rainbow logs with rich :rainbow:
+### Structured & Asynchronous Logging with Rotoger 🪵
 
-To enhance the developer experience when viewing logs with extensive information from multiple emitters 
+To elevate the logging capabilities beyond simple colored output,
-(which are particularly useful during development), this project uses the [rich](https://github.com/Textualize/rich) library.
+this project has transitioned to [Rotoger](https://github.com/tinyplugins/rotoger).
-Event with the superpowers of [rich](https://github.com/Textualize/rich), reading logs can be challenging.
+This powerful library provides a comprehensive, production-ready logging setup for modern asynchronous applications,
-The [rich](https://github.com/Textualize/rich) library is highly beneficial, but integrating it properly as a logger object
+addressing challenges like log management, performance, and readability.
-and maintaining it as a singleton took some effort.
 
-To address the following needs:
+Rotoger is built upon the excellent [structlog](http://structlog.org/) library and brings several key advantages:
-- Difficulty in finding specific information in logs.
-- Avoiding the complexity of setting up an ELK stack for log management.
-- Speeding up the debugging process.
 
-he following steps were taken to integrate [rich](https://github.com/Textualize/rich) into the project:
+- `Structured Logging`: By using structlog, all log entries are generated as structured data (JSON), making them machine-readable and significantly easier to query, parse, and analyze in log management systems.
-1. Configure emitters using the [logging-uvicorn.json](https://github.com/grillazz/fastapi-sqlalchemy-asyncpg/blob/main/logging-uvicorn.json)
+- `Asynchronous & Non-Blocking`: Designed for async frameworks like FastAPI, Rotoger performs logging operations in a non-blocking manner. This ensures that I/O-bound logging tasks do not hold up the event loop, maintaining high application performance.
-   or use [logging-granian.json](https://github.com/grillazz/fastapi-sqlalchemy-asyncpg/blob/main/logging-granian.json) for granian
+- `High-Performance JSON`: It leverages orjson for serialization, which is one of the fastest JSON libraries for Python. This minimizes the overhead of converting log records to JSON strings.
-2. Eliminate duplicates, such as SQLAlchemy echo, by using separate handlers.
+- `Built-in Log Rotation`: Rotoger implements its own log rotation mechanism in Python, allowing you to manage log file sizes and retention policies directly within your application without relying on external tools like logrotate.
-3. Maintain the logger as a singleton to prevent multiple instances.
-4. Add the --log-config ./logging-uvicorn.json parameter to Uvicorn or --log-config ./logging-granian.json to Granian.
 
-![sample-logs-with-rich](/static/logz.png)
+This setup solves common logging pain points in production environments, such as managing large log files, ensuring logs don't impact performance, and making logs easily searchable.
 
 <p align="right">(<a href="#readme-top">back to top</a>)</p>
 

app/utils/logging.py

@@ -1,98 +0,0 @@
-import logging
-import os
-from logging.handlers import RotatingFileHandler
-from pathlib import Path
-
-import orjson
-import structlog
-from attrs import define, field
-from whenever._whenever import Instant
-
-from app.utils.singleton import SingletonMetaNoArgs
-
-
-class RotatingBytesLogger:
-    """Logger that respects RotatingFileHandler's rotation capabilities."""
-
-    def __init__(self, handler):
-        self.handler = handler
-
-    def msg(self, message):
-        """Process a message and pass it through the handler's emit method."""
-        if isinstance(message, bytes):
-            message = message.decode("utf-8")
-
-        # Create a log record that will trigger rotation checks
-        record = logging.LogRecord(
-            name="structlog",
-            level=logging.INFO,
-            pathname="",
-            lineno=0,
-            msg=message.rstrip("\n"),
-            args=(),
-            exc_info=None,
-        )
-
-        # Check if rotation is needed before emitting
-        if self.handler.shouldRollover(record):
-            self.handler.doRollover()
-
-        # Emit the record through the handler
-        self.handler.emit(record)
-
-    # Required methods to make it compatible with structlog
-    def debug(self, message):
-        self.msg(message)
-
-    def info(self, message):
-        self.msg(message)
-
-    def warning(self, message):
-        self.msg(message)
-
-    def error(self, message):
-        self.msg(message)
-
-    def critical(self, message):
-        self.msg(message)
-
-
-class RotatingBytesLoggerFactory:
-    """Factory that creates loggers that respect file rotation."""
-
-    def __init__(self, handler):
-        self.handler = handler
-
-    def __call__(self, *args, **kwargs):
-        return RotatingBytesLogger(self.handler)
-
-
-@define
-class AppStructLogger(metaclass=SingletonMetaNoArgs):
-    _logger: structlog.BoundLogger = field(init=False)
-
-    def __attrs_post_init__(self):
-        _log_date = Instant.now().py_datetime().strftime("%Y%m%d")
-        _log_path = Path(f"{_log_date}_{os.getpid()}.log")
-        _handler = RotatingFileHandler(
-            filename=_log_path,
-            maxBytes=10 * 1024 * 1024,  # 10MB
-            backupCount=5,
-            encoding="utf-8",
-        )
-        structlog.configure(
-            cache_logger_on_first_use=True,
-            wrapper_class=structlog.make_filtering_bound_logger(logging.INFO),
-            processors=[
-                structlog.contextvars.merge_contextvars,
-                structlog.processors.add_log_level,
-                structlog.processors.format_exc_info,
-                structlog.processors.TimeStamper(fmt="iso", utc=True),
-                structlog.processors.JSONRenderer(serializer=orjson.dumps),
-            ],
-            logger_factory=RotatingBytesLoggerFactory(_handler),
-        )
-        self._logger = structlog.get_logger()
-
-    def get_logger(self) -> structlog.BoundLogger:
-        return self._logger