Overview of Long-Term Data Retention in Dataverse for Dynamics 365 CE: Enabling and Managing Data Efficiently



**Overview of Long-Term Data Retention (LTDR)**


- LTDR addresses the need for organizations to manage large volumes of data over time, allowing for the classification of data into active, inactive, and deleted categories.

- It was developed in response to customer requests for a cost-effective way to retain data that is only occasionally accessed for compliance or regulatory reasons.

- Data is moved to a less expensive storage tier within Dataverse, maintaining security and compliance standards without impacting performance.


**Data Management and Security**


- LTDR ensures that data remains within the Dataverse environment, adhering to regulations such as GDPR, as security measures apply to both active and cold storage.

- When data is marked for retention, all related child records are automatically included, simplifying data management.

- The feature allows for easy access to retained data through various methods, including APIs, advanced find, and Power Automate, while maintaining strict security protocols.


**Storage Optimization and Cost Savings**


- By moving inactive data to cold storage, organizations can achieve significant cost savings—often up to 80% due to data compression.

- For example, moving 200 GB of data can reduce storage requirements to as little as 100 GB, resulting in substantial financial efficiency.

- The process is designed to minimize performance impact, allowing for seamless data transitions without affecting transactional operations.


**Enabling LTDR**


- Users with administrative roles can enable LTDR through the Maker experience in Dataverse, selecting specific tables for retention.

- Policies can be set to automate the retention process based on defined criteria, with scheduled frequencies for data movement.

- It is advisable to test retention policies with smaller data sets before full implementation to ensure accuracy and alignment with organizational needs.


**Viewing and Managing Retained Data**


- Retained data can be accessed through Advanced Find, Power Automate, or directly via APIs, ensuring flexibility in data retrieval.

- The management interface provides options for viewing policy history, modifying existing policies, and managing the lifecycle of retention policies.

- Organizations should be aware of limitations in querying retained data, such as restrictions on certain functions and API call limits.


**Best Practices for Implementation**


- Early discussions regarding data retention should be part of project planning, particularly during migrations to ensure compliance and manage costs effectively.

- Regular reviews of retention policies and data access requirements can help organizations stay compliant with evolving regulations and internal policies.

- Users are encouraged to create verification processes before executing retention jobs to avoid unintended data loss or retention errors.

Comments

Post a Comment

Popular posts from this blog

🔍 Dataverse + Azure Integration: Choosing Between Synapse Link and Microsoft Fabric

  Managing large-scale data in Microsoft Dataverse ? You're likely choosing between Azure Synapse Link and the newer Microsoft Fabric Link . Both unlock powerful analytics and reporting options—but with different trade-offs in performance, setup, and long-term cost. Let’s break it down 👇 📊 Power BI and Dataverse: Great, But With Limits Power BI uses TDS (Tabular Data Stream) to connect directly to Dataverse tables. Real-time querying works well for smaller datasets (tens of thousands of records). However, complex queries and relationships may cause timeouts after 5 minutes , making it unreliable for enterprise-scale analytics. 🔗 Azure Synapse Link: Power + Complexity ✅ Strengths: Designed for large datasets , converting Dataverse tables to CSV format for efficient querying and reporting. Highly customizable with Synapse Studio , Azure Data Lake , and Data Factory for deeper ETL/ELT capabilities. ⚠️ Watchouts: Requires technical setup : Azur...
Read more

⚡ Example: Rate Limiting in Azure API Management

  Scenario: You have a public API and want to limit each user to 5 calls per minute to prevent abuse or accidental overload. 🛡️ What is Rate Limiting? Rate limiting controls how many requests a client can make to your API within a specific time window . It helps: Prevent API abuse Protect backend systems Fairly distribute API usage 💡 Using the rate-limit-by-key Policy Azure APIM uses a policy called rate-limit-by-key . It limits requests based on a unique identifier — usually the caller's subscription key or IP address. 🔧 Example Policy (XML) Here’s how you can add this to your inbound policy : < inbound > < base /> < rate-limit-by-key calls = "5" renewal-period = "60" counter-key = "@(context.Subscription?.Key)" /> </ inbound > Explanation: Attribute Meaning calls="5" Maximum 5 calls allowed renewal-period="60" Time window of 60 seconds (1 minute...
Read more

👤 Anonymous Role in Power Pages – What It Is and When to Use It

  When building public-facing websites with Power Pages , not every visitor will be a logged-in user. Some might just be browsing, some might want to ask a question, and others might be using your portal for the very first time. So how do we allow unregistered users to interact with our portal safely ? That’s where the Anonymous Users role comes in. 🚪 Who Are Anonymous Users? Anonymous users are visitors who haven’t logged in to your Power Pages portal. They: Have not signed in May not have an account Might be visiting for the first time And here’s the best part — Power Pages automatically assigns them the “Anonymous Users” web role. You don’t have to do anything special to give it. 🔐 What Can They Do? By default, anonymous users can see pages marked as "anonymous access allowed" , but they can't create or edit data unless you let them . To let anonymous users interact with your data (e.g., submit a form), you need to: Create table permissi...
Read more