Database and BI
ETL (transformation avant chargement)
ELT : chargement dans le Data Warehouse/Lakehouse avant transformation
La gouvernance des données désigne l'ensemble des règles, politiques et processus qui permettent à une organisation de gérer, sécuriser et valoriser ses données tout au long de leur cycle de vie. [1, 2]
Les 4 piliers fondamentaux
Qualité : Garantir que les données sont fiables, exactes et à jour.
Sécurité et confidentialité : Protéger les données contre les accès non autorisés et les pertes.
Conformité : Respecter les lois en vigueur, comme le RGPD ou les normes ISO.
Disponibilité : S'assurer que les collaborateurs autorisés peuvent accéder aux bonnes données au bon momen
Microsoft Fabric is an end-to-end, Software as a Service (SaaS) unified data analytics platform. It seamlessly combines data engineering, data integration, warehousing, data science, and business intelligence into a single, collaborative workspace. [1, 2, 3]
Core Components
• OneLake: Often described as the "OneDrive for data," it serves as a single, unified, logical data lake for your entire organization, eliminating the need to move data between different tools.
• Data Factory: Provides robust capabilities for complex data ingestion, preparation, and orchestration.
• Data Engineering & Science: Utilizes Apache Spark-powered notebooks and engines to build, train, and deploy machine learning models.
• Data Warehousing: Delivers high-performance, schema-based data warehousing within a unified ecosystem.
• Real-Time Intelligence: Simplifies working with streaming data from IoT devices and applications.
• Power BI: Deeply integrated into the platform to allow users to visualize insights and build interactive reports seamlessly on top of OneLake
•
La conception d'un entrepôt de données (ou Data Warehouse) est le processus de planification et de structuration d'un système centralisé permettant de regrouper, d'harmoniser et d'analyser de grands volumes de données issues de sources multiples pour l'aide à la decision.
Date modeling :
. Conceptual Data Modeling: A high-level, business-oriented view identifying key entities and relationships, often termed domain models.
. Logical Data Modeling: Maps business rules to technical structures (tables, columns, keys) without specific database technology details.
. Physical Data Modeling: Defines specific database implementations, including column types, indexes, and partitions, tailored to a chosen DBMS (e.g., PostgreSQL, Snowflake).. . Dimensional Modeling (Kimball): Optimized for data warehousing and analytics using star schemas (fact and dimension tables) to enable fast querying and reporting.
Benefits of BI Solution Architecture
Enhanced Decision-Making: Provides accurate, real-time insights for informed decisions.
Operational Efficiency: Automates data processes, reducing manual errors.
Scalability: Adapts to growing data volumes and evolving business needs.
Data Governance: Ensures data accuracy, security, and compliance.
