Montevideo 14 de Diciembre

Sácale provecho a tus datos con 64 bits y OLAP Mariano Alvarez mariano.alvarez@sqlgurus.org Christian Linacre christian.linacre@microsoft.com

Sácale provecho a tus datos con 64 bits y OLAP Migración de DTS SQL Server 64 bits SQL Server Integration Services Analysis Services - OLAP Analysis Services - Mining

Plataforma integrada de administración de datos Talking points: SQL Server is a comprehensive, integrated end to end data platform including: Core database engine: secure, reliable, highly available data store Data transformation and replication services (DTS, Replication) facilitate enterprise wide data integration, data warehouse creation, data replication for distributed or mobile data processing applications, and systems availability Deriving additional value from your data: Notification Services, Data Analysis, Data Mining, Reporting Management tools – SQL specific management suite as well as tight integration with tools such as MOM, SMS Development tools – BI specific development toolset as well as tight integration with Visual Studio Flexible, extensible: Every major subsystem in SQL Server ships with it’s own Object Model and set of APIs to you to extend the data system in any direction that is unique to your business Interoperable: Standard data access protocols drastically reduce the time it takes to integrate data in SQL Server with existing systems. In addition, native web service support is built into SQL Server to ensure interoperability with other applications and platforms SQL Server is the only database to include all of this functionality out of the box. Competitive offerings rely on third part applications to address key functionality and/or charge for incremental functionality (such as BI) Also provide entry level SKUs for lightweight data needs (Express, Workgroup) Scales to your needs: support for databases from the mobile device to the datacenter SQL Server Mobile, 32 bit, 64 bit Hardware platform range of SQL Server includes: Single Pocket PC CPU systems, Throughout mobile and desktop class PCs, Through Multi-CPU server systems All the way up to 64-processor Enterprise Data Center systems

Versiones de SQL Server 2005 Express Estudiantes Workgroup Pymes Standard Medianas empresas – Aplicaciones Departamentales Enterprise Grandes empresas – Misión crítica

SQL Server 64 bits SQL Server 2005 es para 64 bits Plataforma Windows 2003 Server 64 bits x64 Itanium Aprovecha el aumento de capacidad de cómputo de la nueva arquitectura Ideal para escenarios: Alto rendimiento Integración de datos Análisis de datos Concurrencia (Snapshot Isolation) Tablas particionadas

64 bits Mariano Alvarez

Arquitectura de integración de datos Antes de Integration Services Alertas y escalamiento Data mining Datos Call center: semi estructurados Datos Legacy: archivos binarios Base de datos de aplicación Codificación Manual Staging Mining de Texto ETL Limpiado de datos y ETL Staging ETL Warehouse Reportes Datos Móviles Integración y el warehouse requieren operaciones y staging separados Preparación de los datos requieren herramientas diferentes Reportes y escalamiento es un proceso lento, demorando las respuestas Altos volumenes de datos hacen que el escenario sea casi imposible de trabajar

Data Integration Architecture With Integration Services Arquitectura de integración de datos Con Integration Services Alertas y escalamiento Warehouse Reportes Datos Móviles SQL Server Integration Services Componentes Mining de texto Fuente Personalizada Estandar Componentes limpiado de datos Merges Data mining Call center: Datos Semi-structurados Datos Legacy: archivos binarios Base de datos de aplicación Integración y warehouse son de operación y administración simple Todo el flujo de datos en un proceso único y auditable Reportes y escalamiento pueden ser paralelizados con la carga del warehouse Escala para manejar altos y complejos volúmenes de datos

Integration Services Mariano Alvarez

Analysis Services ¿Por qué OLAP y Data Mining? Una versión de la verdad Modelamiento de la información de negocios Integración de datos multi plataforma Vistas relacionales y OLAP integradas Análisis avanzado y valor sobre los datos KPI: Key Performance Indicators y Perspectivas Tiempo real, alto rendimiento Datos en tiempo real en cubos OLAP Muy rápido y analisis flexible Estandar XML para acceso a datos e integración con Web Services Ahorro de tiempo y dinero al integrar clientes con otros sistemas The Analysis Services technology remains at the heart of Microsoft’s BI capabilities. It is now in its third generation and has delivered a wide set of new options – both functional & management/scalability focused. We’ll focus on three key areas: The UDM – a new approach to modeling the inputs to the OLAP capabilities of SQL Server Analysis Services that can help eliminate costly data staging areas and business-function specific data marts KPI – a new architecture for delivering goal-based metrics to the organization Deep Data Mining – moving beyond slice & dice and drilldown to provide tools that help you catch complex relationships and patterns in your data and predict outcomes based on past results.

Analysis Services OLAP mejorado y capacidades de Data Mining UDM: Unified Dimensional Model Pro-active caching Business Intelligence avanzado KPI/Perspectivas Agregaciones personalizadas y métricas semi-aditivas Web services Data Mining en la plataforma Herramientas de desarrollo integradas Soporta Clustering y multi-instancias The Analysis Services technology remains at the heart of Microsoft’s BI capabilities. It is now in its third generation and has delivered a wide set of new options – both functional & management/scalability focused. We’ll focus on three key areas: The UDM – a new approach to modeling the inputs to the OLAP capabilities of SQL Server Analysis Services that can help eliminate costly data staging areas and business-function specific data marts KPI – a new architecture for delivering goal-based metrics to the organization Deep Data Mining – moving beyond slice & dice and drilldown to provide tools that help you catch complex relationships and patterns in your data and predict outcomes based on past results. más… Regresión logística Regresión Lineal Mining Texto Series de tiempo Clustering de secuencia Asociación Red Neuronal Arboles de decisión Clustering Naïve Bayes Introduced in SQL Server 2000

Analysis Services Arquitectura de alto nivel Planillas SQL Server Datamart XML/A or ODBO UDM Presentación BI Teradata DW Reportes Ad Hoc ODBO = OLEdb for OLAP XMLA = XML for Analysis Reports Oracle DB2 LOB Cache Dashboards

SQL Server Analysis Services Nuevo paradigma para la plataforma Unified Dimensional Model Modelamiento de datos de negocios Integración multi-plataforma Vistas relacional y OLAP integradas KPIs y Perspectivas Proactive caching Datos en tiempo real en cubos OLAP Análisis rápido y flexible Mejoras de Business Intelligence Add data-aware “smarts” Autogeneración KPIs, MDX scripts, traducciones, monedas… Data Mining 10 Algoritmos de Mining Aplicaciones inteligentes Estandar XML para acceso a datos e integración con Web services Ahorro de costos para los clientes al integrar esta solución con otros sistemas UDM: SQL Server 2005 introduces the Universal Data Model – this technology provides the mechanism to describe data sources in a BI-friendly manner without requiring changes to the source data. This can eliminate the need for staging areas as data can be consumed directly from the source systems. The models can then be used to drive multiple cubes (or live caches of the underlying data). By looking at the cubes as a series of high-performance caches the best of the OLAP & OLTP worlds can be combined. Cache: The cache is a MOLAP datastore that manages the retrieval of data from backend data sources. You can control how frequently the multidimensional cache is rebuilt, if stale data can be queried while the cache is being refreshed, and whether data is retrieved to a schedule or when it changes in the database. Business Intelligence Smarts: SQL Server 2000 included time dimension awareness – this is extended in SQL Server 2005 to cover autogeneration of several key calculations that really help you to jumpstart any BI system: Time – adds the following calculations: Period to date, Period over period growth, Moving average and Parallel period comparisons. Accounting – cost, balance & other accounting calculations Dimension – choose from a list of known dimension types and also automatically set additional attributes to autogenerate appropriate calculations Set your own aggregation operator or calculation and control updateability Data Mining: We’ll cover more on this later XMLA: SQL Server 2005 implements the open standard: XML for Analysis – optimized for scalable web access to access & define OLAP data.

Business Intelligence Obteniendo lo importante “Parálisis por Análisis” El riesgo de proveer data sin procesar o grandes volumenes de datos Paradoja: es importante tener el detalle para entender el origen de los datos BUILD {BASE} Causality : n : the relation between causes and effects If a manager is presented with a banded report containing hundreds of pages of small-font data points its likely that trends, spikes & troughs will be missed; even if presented to the manager in an Excel workbook there is a risk that their own calculations will miss important changes and trends in the data. However it is important to have this detailed level of data available for the cause behind changes in key business measures {KPI} SQL Server 2005 introduces a facility to define Key Performance indicators that can add velocity to data KPI

OLAP Mariano Alvarez

El valor del Data Mining Conocimiento del negocio Fácil Díficil Usabilidad Valor relativo de negocios SQL Server 2005 Data Mining OLAP Reportes (Ad Hoc) Reportes (estáticos) BUILD {BASE} Data mining has long been regarded as extremely hard to do properly (the domain of high-paid Wall Street “rocket scientist” statisticians) but conversely as having huge potential business value because of its abilities to spot clusters and trends in large data volumes that a human would miss. {STATIC REPORTS} Are easy, predefined, canned reports with no inputs – they are a great operational tool but are unlikely to deliver new business insight {AD HOC REPORTS} Add more value as start to introduce drill-down-like capabilities, empowering data exploration, however they still tend to be limited in terms of navigating predefined relationships {OLAP} Represents a big leap in business value – drill through, slicing and dicing to compare data and the new capabilities of SQL Server 2005 provide real business insight {DATA MINING, pause then click to “move the arrow”} Long regarded as hard, SQL Server 2005 provides multiple technologies to ease its development, deployment and use 8 nuevos algoritmos, 10 en total Asistentes y herramientas gráficas 12 visores de datos SQL Server 2005 lo hace más fácil Fuertemente integrado con AS, DTS, Reporting Integración con aplicaciones Web/Office

Conjunto de Algoritmos Arboles de decisión Clustering Series de tiempo Naïve Bayes Introduced in SQL Server 2000 Clustering de secuencia Asociación Redes neuronales Regresión Logística All data mining tools, including Microsoft SQL Server 2005 Analysis Services, use multiple algorithms. Analysis Services, of course, is extensible; third party ISVs can develop algorithms that snap in seamlessly to the Analysis Services data mining framework. Depending on the data and the goals, different algorithms are preferred, and each algorithm can be used for multiple problems. Regresión Lineal Mining de Texto

Cómo usar Mining Tarea Algoritmo Microsoft Predecir un atributo discreto. Por ejemplo, para predecir cuando el recipiente de un mailing dirigido compra un producto Decision Trees Naive Bayes Clustering Red Neuronal Regresión Logística Regresión Lineal Predecir un atributo continuo. Por ejemplo, para proyectar las ventas del próximo año Árboles de decisión Series de tiempo Predecir una secuencia. Por ejemplo, para analizar el “clickstream”en el sitio web de la compañía. Clustering de secuencia Encontrar grupos de itemes comunes en transacciones. Por ejemplo, analizar el análisis de canasta de productos para sugerir productos adicionales para comprar a un cliente. Reglas de asociación Encontrar grupos de itemes similares. Por ejemplo, para segmentar datos demográficos en grupos para entender las relaciones entre los atributos.

Mining Mariano Alvarez

Resumen SQL Server 2005 integra las plataformas de negocio SQL Server Integration Services provee Soluciones de manipulación de datos Integración de ambientes heterogéneos SQL Analysis Services OLAP para construir modelos dimensionales Mining para encontrar los datos “valiosos”

Siguiente Charla ¿Son tus datos inteligentes? Christian Linacre Charla Intel