darwin.Cloud audit log isolation

The Critical Role of Audit Logs

Audit logs are a cornerstone of our darwin.Cloud, providing a detailed record of every data change within the system. These logs are particularly crucial for:

• Financial and Sales Transactions: Offering an exact history of who did what, when, and how.
• Compliance and Accountability: Ensuring adherence to regulations and enabling oversight.
• Post-Event Analysis: Supporting investigations and answering questions like, “What happened and why?”

While audit logs are indispensable for research and accountability, they are rarely used in real-time processes. Instead, their primary value lies in after-the-fact analysis, such as reviewing past transactions or diagnosing issues.

The Challenge

Historically, audit logs in darwin.Cloud were stored on the same production servers that users rely on for daily operations, such as viewing records and inputting new data. This setup presented challenges:

1. Increased Workload: Generating and retrieving audit logs places strain on the production servers, which also handle core user activities.
2. Performance Impact: Demand on the servers for audit log updates and queries can slow down other critical operations.
3. Missed Optimization Opportunity: Since audit logs are not time-sensitive, there was no need for them to be managed in real-time alongside primary user data.

Key Benefits for Our Users

The Solution: Dedicated Infrastructure for Audit Logs

To address these challenges, we are implementing a new approach that offloads audit logs to dedicated hardware, completely separate from the production servers. Here’s what we are done:

1. Moved Audit Logs to Separate NVMe Drives on a Distinct SQL Server

• What We Did:
  • All audit logs are will be hosted on high-performance NVMe drives located on a separate SQL Server, optimized specifically for read and write operations related to audit data.
• The Benefit:
  • This setup ensures that audit log operations no longer impact the production server’s performance.

2. Redirected Application Requests to the New Audit Log Server

• What We Did:
  • All parts of the application that need to retrieve or update audit logs will now interact with this dedicated server.
• The Benefit:
  • This prevents the production server from handling any audit log workloads, allowing it to focus entirely on user operations like retrieving and inputting data.

3. Offloaded Reporting Workloads

• What We Did:
  • Updates and queries that include audit log data will now run against the dedicated server, rather than pulling data from the production server.
• The Benefit:
  • By removing this overhead, the production server remains optimized for its primary purpose, while audit logs remain fully accessible for reporting and analysis.

Why This Matters to You

This enhancement brings several significant benefits for our users:

1. Faster Day-to-Day Operations:
   • By removing the burden of audit log management from the production servers, users will experience faster response times for core activities like viewing records and entering data in darwin.Cloud.
2. Reliable and Accessible Audit Logs:
   • The dedicated infrastructure ensures that audit logs remain available and secure, supporting detailed research and compliance needs.
3. Improved Reporting Performance:
   • Reports that include audit data now run more efficiently, thanks to the separation of workloads between production and audit log servers.
4. Optimal Use of Resources:
   • Hosting audit logs on dedicated hardware allows us to leverage the right tools for the right tasks, improving overall system performance.

Balancing Criticality and Timeliness

While audit logs are indispensable, they are not time-sensitive in darwin.Cloud. This understanding guided our decision to optimize their hosting without compromising their integrity. By prioritizing the performance of real-time user operations while ensuring robust access to audit logs for post-event analysis, we’ve achieved the best of both worlds.

Conclusion

At AccountTECH, we recognize the critical role that audit logs play in maintaining transparency, accountability, and compliance. By redesigning our infrastructure to host these logs on dedicated servers, we will not only enhanced darwin.Cloud performance but also ensured that audit logs remain accessible for the vital research and reporting tasks they support.

This project reflects our commitment to continuously improving the technology that powers your business, delivering solutions that are both reliable and efficient.