Estudio de casos similares a los requeridos por los TdRs de la consultoría Emiliano Marmonti Bogotá. Colombia 27 de Agosto de 2012.

Presentación del tema: "Estudio de casos similares a los requeridos por los TdRs de la consultoría Emiliano Marmonti Bogotá. Colombia 27 de Agosto de 2012."— Transcripción de la presentación:

1 Estudio de casos similares a los requeridos por los TdRs de la consultoría Emiliano Marmonti Bogotá. Colombia 27 de Agosto de 2012

2 Finalidad de este espacio Dado que es espíritu de la propuesta presentada re-aprovechar las experiencias anteriores y en curso existentes e idealmente sumarse a un desarrollo activo viable en curso, es que en el presente espacio se revisarán antecedentes internacionales en cuanto a soluciones similares.

3 Portal DRIVER +/- 6.5 M registros

4 The DRIVER Project: Building a European Repository Network Library Science Talks series Geneva/Bern, 7/8 December 2008 Rosemary Russell UKOLN, University of Bath

5 Project partners University of Athens (GR) University of Bielefeld (DE) CNR-ISTI (IT) SURF Foundation (NL) University of Nottingham (UK) UKOLN, University of Bath (UK) University of Warsaw (PO) University of Gent (BE) University of Gottingen (DE) Danish Technical University (DK) National and University Library (SLO) University of Minho (PT)

6 DRIVER vision All research institutions in Europe and worldwide make their research publications openly accessible through institutional repositories (Norbert Lossau, University of Bielefeld, DRIVER Scientific Coordinator)

7 DRIVER primary objectives to build a virtual, European-scale OA network of existing institutional repositories to develop open source state-of- the-art infrastructure software D-NET 1.0 released

8 Adapted from original presentation by Norbert Lossau, DRIVER Scientific Coordinator, delivered at SPARC Conference, Baltimore 2008 DRIVER II activity areas Open Source Software for repository networks Technical Digital Repository Network Infrastructure Services Organization of Digital Repository Infrastructure Providers Community Building & Maintenance Confederation D-NET DR Infrastructure DRIVER Portal & Wiki

9 D-NET (DRIVER Network Evolution Toolkit) Open source software Version 1.0 released under Apache License on 20 June 2008 Software includes: Repository network administration software (eg Repository Network Manager, Resource Monitoring) End-user services (Search, Browse, Profiling) Support service to local repository managers and aggregators (Validation Tool)

10 DRIVER Infrastructure First implementation of D-NET - DRIVER European Information Space (DEIS) - maintained by the DRIVER Consortium Supports 3 groups of users: 1. Repository managers 2. Service providers 3. Researchers, general public

11

12 DRIVER search Free to use, no access control Search, browse Personalisation possible – create profile Search communities, collections

13

14 Repository registration Target Group Repository Managers Policies Registration 1.Open / through website 2.Recommendation by country correspondent 3.Pro-active inclusion by DRIVER DRIVER Guidelines / Validation process

15 DRIVER Guidelines Needed for building high-quality services Standardise record quality in harvested IRs Based on DINI-certificate, DAREnet, ePrints UK Very lightweight v.2 just published Guidelines include: OAI-PMH protocol guidelines (eg use of sets that identify collections of full-text – since not all full text…) Metadata requirements for repositories (since Dublin Core interpreted in different ways…)

16 Validation tool – why? Automation of DRIVER guidelines Self-test for repository managers via portal Inclusion in clean aggregration Added value services build only on validated records Influence ranking of search results NB: Not essential to comply with guidelines to join DRIVER, but provide general good practice for IRs anyway

17 An open infrastructure… D-NET software can be configured and used by other communities to deploy new services on top of existing infrastructure to deploy a new DRIVER infrastructure to serve other communities Configured services can run in same environment applications can share services Recolecta, DART - access subset of info space

18 D-NET configuration: Recolecta

19 D-NET configuration: DART Europe

20 D-NET configuration: Belgium

21 Next version of D-NET… Enhanced publications Type of complex object Publication with additional materials eg datasets, images, video, software, annotations Explicit links between components (each has own persistent identifer) Demonstrator of EPs - uses OAI-ORE Technology watch, includes: Interoperability [eg ISO OOXML…] CRIS Long term preservation

22 Mentor service For repository managers/developers Peer support – one-to-one advice and guidance Share best practice and experience Potential mentees submit request to DRIVER mentor service: http://www.driver-support.eu/mentor.html Mentor service identifies mentor with relevant expertise (if consider that existing online resources cannot meet needs)

23 DRIVER Portal and Wiki Portal Access information and support www.driver-community.eu Wiki Contribute to DRIVER community www.driver-support.eu/pmwiki

24 DRIVER Portal

25 DRIVER wiki

26 What infrastructures are: DRIVER terms Not an infrastructure Single repository Single application for search and retrieval (e.g. BASE) Only local operation Backwards causation on repositories is missing Maybe an infrastructure Distributed repository landscape as a whole As a capacity for emergent properties, e.g. quality and quantity incentive for data population Nurturing development of service providers Definitely an infrastructure Many service providers in one organisational and technical context (e.g. run-time environment) Enabling re-use and remix of data and services

27 DRIVER Objectives Organisational structure for repositories e.g. the Confederation Improving quality and standards in local rep. e.g. validation procedures Building a distributed runtime system e.g. service and data sharing Target Groups Repository Managers Service Providers Information System Executives

28 The DRIVER approach is incremental Start with publication metadata Existing distributed system, somehow connected Considerable homogeneity and formats: OAI-PMH Extend geographical coverage From 5 countries, to 10, to 27, to ??? Extend towards other contents From publication metadata to enhanced publications, i.e. representations of texts + data Learn about subject specificity Data bring in disciplinary requirements

29 29 The DRIVER Initiative DRIVER-I 6/2006 – 11/2007 Organisational Models and Technical Test-Bed DRIVER-II 12/2007 – 11/2009 Running Organisation and Production Infrastructure DRIVER-Confederation 2010ff Operations Office and Technical Deployment NB: DRIVER is not an authoritative body, it is a liberal bottom-up initiative of stakeholders

30 DRIVER partners and related projects Networking, Support, Policy, Studies Göttingen, Nottingham, SURF, Genth, Ljubiljana, Minho, Copenhagen Technical development and deployment Athens, Bielefeld, Pisa, Warsaw Partners make links to many other things OA-services: Sherpa-ROMEO, OpenDOAR, BASE… Projects: Europeana, PEER, DELOS, DL.org, D4Science, PARSE-Insight, NESTOR… Orgs: DINI, JISC, LIBER, SPARC, KE … Platforms: DSPACE/FEDORA/OPUS/ePrints

31 31 DRIVER-II Midterm Review, January 30, 2009 - Pisa 31 Project structure Networking ResearchService Running Infrastructure: Content & Functionality Construction of Services: ideas, design, development Technical Management Advocacy: attracting users, content and Service providers Discovery: technology watch, EPs requirements

32 Some results

33 Some Results: Studies

34 Some Results: Service- Oriented-Arch. 9 hosting nodes 25+ Functionality typologies (services) 36 service Instances 3 applications: DRIVER Main, Belgium, Spain-Recolecta

35 35 Some Results: Runtime-System & Hosting Enabling Layer Data Layer EU Open Access Repositories Functionality Layer Administrators End users Advanced User Interfaces National portals Project Applications

36 Another Compulsory Design Diagram

37 Some Results: A software Meant for large service providers only!

38 Technicalities

39 DRIVER and standards Service Resources are implemented as Web Services and accessed through the corresponding Web Service Interface Parameters calls are enveloped into SOAP messages The Enabling Services are also compatible with REST XML is the lingua-franca for the whole system Resource internal status, i.e. Resource profiles Profiles in Information Service use Exist XML engine Vocabularies Names of Languages: ISO 639 – 2 (three letters, B/T) Names of Countries: ISO 3166 (two letters) Date format: ISO 8601: 1988 (E) DRIVER Aggregation Harvesting according to OAI-PMH protocol Adopting OAI-Provenance best practice (OAI-about) To be extended to other object models and harvesting protocols Queries to Search and Index obey to SRW/CQL standard

40 40 DRIVER-II Midterm Review, January 30, 2009 - Pisa 40 Enabling Layer Developments FunctionTaskPartnerStatusD-NET IS-StoreResource profile storeEnhanced Port (PERL > JAVA) CNRRC1.1 IS-S&NW3C S&N/TopicsEnhanced Port (PERL > JAVA) CNRRC1.1 IS-LookupResource discoveryEnhanced Port (PERL > JAVA) CNRRC1.1 IS-RegistryResource registration/de- registration/update Enhanced Port (PERL > JAVA) CNRRC1.1 ManagerOrchestration of DRIVER Info Space Enhanced Port (PERL > JAVA) CNRRC1.1 Authn&AuthzService-2-Service secure interaction/multiple applications Enhanced Service (JAVA) ICMProto2.0 MonitoringAdmin User Interface and autonomic administration Novel Service (JAVA)CNRRC1.2

41 41 DRIVER-II Midterm Review, January 30, 2009 - Pisa 41 Data-Layer Developments FunctionTaskPartnerStatusD-NET HarvesterCollects arbitrary formatsPort (PERL > JAVA)UniBi/CNRAlpha2.0 TransformatorEases arbitrary mappingsNovel service (JAVA)UniBi/CNRAlpha2.0 Feature ExtractionExecutes transform.s. and utilities Novel service (JAVA)UniBiAlpha2.0 Text-EngineUtilities, e.g. language detection, full-text-extr. Novel service (JAVA)UniBiAlpha1.1 MD-StoreSupport special MD operations Port (PERL > JAVA)UniBiAlpha1.1 StoreGeneric store for binariesNovel service (JAVA)UniBi/ICM/ CNR Proto2.0 IndexLookup table for stored information Adapt from YADDAICM/UniBiProd.1.0 OAI-ORE PublisherExposure of stored information Novel service (JAVA)CNRSpec.2.0 OAI-PMH PublisherExposure of stored information --CNRProd.1.0 Content ServiceManaging complex objectsNovel service (JAVA)CNRProto2.0 Access ServiceGeneric service for using remote objects Novel service (JAVA)CNRProto2.0

42 42 DRIVER-II Midterm Review, January 30, 2009 - Pisa 42 Functional Layer Developments FunctionTaskPartnerStatusD-NET AIDEnhanced Publications management Novel Service (JAVA)NKUASpec.2.0 Advanced searchOptimized Search Similarity Search Enhanced Service (JAVA) Novel Service (JAVA) NKUA ICM Spec. 2.0 User ServicesAdvanced personalizationEnhanced Service (JAVA) NKUASpec.2.0 Community ServiceAdvanced Community management Enhanced Service (JAVA) NKUASpec.2.0 Web InterfaceGeneric to data model and services Enhanced UIs Enhanced Service (JAVA) NKUASpec. Spec 1.2 2.0

43 43 Current Work: DRIVER-II Networking Confederation with who-is-who advisory board Outreach: LIBER, SPARC, US, JAPAN etc… Consolidation DRIVER-I Services packaged and performing in production quality Enhancement DRIVER-I Services Improved indexing and data aggregation functionalities DRIVER-II Services: D-NET v2.0 Enhanced publication management and functionality

44 DRIVER II – D-NET v2.0 Studies What are Enhanced Publications? >> PDFPDF Technologies for Enhanced Publications >> PDFPDF Long-Term Preservation of Enhanced Publications Technology Watch: the Future >> PDFPDF Demonstrators Enhanced Publications >> LiveLive Enhanced Publications Long-Term Preserv. >> FilmFilm Infrastructure Specs. ready, Development in progress >> WIKIWIKI D-NET v1.1: Java-Porting & Build-System D-NET v1.2: New Aggregator, Installer (, Contracts) D-NET v2.0: Compound Object Management

45 Outlook: Enhanced Publications

46 Based on OAI-ORE

47 The Web-Capable Model – OAI-ORE http://www.openarchives.org/ore/

48 The Document Model for DRIVER

49 The Object Model – Internal Processing Primitives: Types, Sets and Objects Object: atoms, descriptions, relations

50 50 The DRIVER-application

51 Compound Object Management Object InstancesDRIVER Processing DRIVER Application Web-Representation Web-Processing

52 Conclusion

53 Lessons learnt Distributed data infrastructure requires links between organisational and technical concepts Data specialists, computer scientists, service providers Guidelines / content policies as a glue In distributed data provision, quality and access measures are the most expensive tasks Distributed service operation (not data provision) can be solved but asks novel questions (SLAs) Infrastructure for novel paradigms for scholarly communication are hard to get across ;-)

54 Summary DRIVER tackles the data infrastructure challenge from the text- repository side (mostly OAI-PMH) DRIVER handshakes with primary & secondary data through enhanced publications DRIVER isnt only a project but a forum for information specialists Products include: Studies, Infrastructure run-time-system in production, software, support … DRIVER has adressed many problems for data and service interoperability in a distributed repository environment and found some solutions

55 But… How could DRIVER link to serious processing of unstructured data?

56 DRIVER Confederation of European Repositories An organisation which will exist beyond the duration of the DRIVER project Initial focus on Europe, but with international alliances from the beginning Confederation partner types: Consortium – funded by DRIVER project Network – provide content Correspondent – various eg software providers, subject communities, research/funding organisations Launch planned for October 2009 Advisory Board set up (LIBER, Istanbul) Need to establish a new, independent legal body or merge with an existing organisation?

57 How does DRIVER relate to national repository communities? National communities are usually represented by country correspondents One institution or a group of institutions takes responsibility to build a national repository community (eg DARENet, SHERPA, OA-Netzwerk, Recolecta, HAL) Country correspondents may eg: Maintain national repository information on the DRIVER wiki Organise repository events in their countries Translate repository guidelines and other relevant information into national languages Build up national data aggregators, clean data, offer additional services

58 How does DRIVER relate to international repository communities? DRIVER sees its mission as Catalyst for a global repository infrastructure Develop and maintain the European repository infrastructure node DRIVER already liaises with institutions and initiatives from the majority of European countries, the US, Canada, Latin America, China, Japan, India and Africa DRIVER has signed MoUs with SPARC Europe, LIBER, eIFL, Recolecta Spain, OA- Netzwerk Germany and DRF Japan

59 Find out more at OAI6… DRIVER tutorial @ OAI6, Geneva 17 June 2009, 0900 – 1130 Programme focus on how to get involved in DRIVER, to include: Confederation Registration/validation Guidelines for data providers Tools for service providers Mentor service

60 Caso Portugal RCAAP (2009) +/- 450K regs

61 D-NET and the Portuguese Repository Portal (RCAAP) DRIVER Summit, 20 th October 2009, Ghent Eloy Rodrigues Director Documentation Services Universidade do Minho

62 Building a national repository network RCAAP (Repositório Científico de Acesso Aberto de Portugal = Portugal Open Access Science Repository) www.rcaap.ptwww.rcaap.pt Main components of RCAAP: RCAAP Portal A national aggregator and search portal for repositories Currently 19 repositories harvested 30.000+ documents SARI (Serviço de Alojamento de Repositórios Institucionais) A repository hosting service (ASP) for Portuguese research institutions without IRs (based in Dspace) Currently 10 repositories hosted (+4 in installation phase)

63 RCAAP Portal

64 DRIVER and RCAAP RCAAP Portal/Aggregator will shortly require DRIVER Guidelines (V2) compliance to aggregate repositories RCAAP Portal was planned to be created in 2008 with D-NET v1.0, but… complexity of software and some problems detected, completely new D-NETversion planned/announced and lack of resources from DRIVER consortium to offer helpdesk service and support on the installation and configuration process, …led us to choose another solution: develop portal based on ARC, and a local validator

65 Evaluating D-NET for RCAAP Portal 2.0 RCAAP planned to have a new version of the Portal by the end of November 2009, with some added features (OAI Data Provider, Full Text indexing, etc.) In July/August D-NET 1.1 was installed, tested and comparatively evaluated with the current RCAAP platform (homegrown, based on ARC) for the development of RCAAP 2.0 This process was supported by the team of NKUA (Antonis Lempesis)

66 Evaluating D-NET for RCAAP: Conclusions Advantages of DNET Provides more features/services than a simple aggregator; Very flexible and modular Would facilitate integration/interoperability with DRIVER European Information Space Disadvantages of DNET Poorly documented (installation and usage) Insufficiently mature The requirements in terms of infrastructure are highly incompatible with what is currently developed under the project RCAAP

67 Evaluating D-NET for RCAAP: Conclusions The costs of adapting D-NET… adapting interface to RCAAP identity developing new features adapting RCAAP infrastructure to D-NETs requirements …would be considerably higher than updating/developing the current platform to the new features So, we decide to develop RCAAP 2.0 with the current platform

68 Thank you, for your attention www.rcaap.pt

69 Caso Latindex

70 Portal de Portales Latindex Sistema de prueba Guillermo Chávez Sánchez Octubre de 2009

71 71 Dirección General de Servicios de Cómputo Académico / DSSI / Publicaciones Digitales//Octubre 2009 71 Antecedentes : Proyecto surgido de una iniciativa en la reunión técnica de Latindex del 2007. La idea naciente fue ofrecer nuevos servicios a través de Latindex orientados a un sector de la población mucho más amplio, sin perder la esencia del manejo de la información sobre revistas producidas en la región. En el 2008, en la reunión técnica de Latindex en Panamá se planteó el proyecto que mostrará no sólo la información bibliográfica, sino los registros catalográficos y los textos completos de todos los artículos en línea de la región denominando al proyecto Portal de Portales, Latindex.

72 Dirección General de Servicios de Cómputo Académico / DSSI / Publicaciones Digitales//Octubre 2009 72 Un portal, a través del cual, el usuario pueda consultar, a texto completo, todos aquellos artículos publicados en las revistas editadas en la región que se encuentren disponibles en Internet. Portal de Portales, Latindex es:

73 Dirección General de Servicios de Cómputo Académico / DSSI / Publicaciones Digitales//Octubre 2009 73 La búsqueda de la información por parte de los usuarios será a través de Metadatos Portal de Portales, Latindex es:

74 Dirección General de Servicios de Cómputo Académico / DSSI / Publicaciones Digitales//Octubre 2009 74 Objetivos planteados: General: Proporcionar un portal de acceso organizado a los contenidos publicados en revistas académicas iberoamericanas disponibles en otros sitios (repositorios, bases de datos o hemerotecas virtuales) a nivel de texto completo, resúmenes, tablas de contenido o registros bibliográficos.

75 Dirección General de Servicios de Cómputo Académico / DSSI / Publicaciones Digitales//Octubre 2009 75 Objetivos planteados: Particulares: Ofrecer al usuario de forma ágil y sencilla la posibilidad de consulta de la información bibliográfica de los artículos de todas las revistas iberoamericanas disponibles en Internet. Ofrecer al usuario de forma ágil y sencilla acceso organizado a los contenidos en formato electrónico de todas las revistas iberoamericanas disponibles en Internet. Posibilidad de realizar análisis sobre la producción de revistas académicas de Iberoamérica de acceso abierto. Posibilidad de analizar los procesos y metodologías para la catalogación de los recursos electrónicos de revistas disponibles en línea en la región.

76 Dirección General de Servicios de Cómputo Académico / DSSI / Publicaciones Digitales//Octubre 2009 76 Etapas del proyecto Proyecto planteado en 5 etapas Situación actual del proyecto

77 Dirección General de Servicios de Cómputo Académico / DSSI / Publicaciones Digitales//Octubre 2009 77 Herramientas utilizadas en el prototipo Sistema operativo Linux Ubuntu 9.04 Open Archives Harvester 2.0 (cosechador de metadatos) PHP 5.0 Servidor Web Apache 2.0 SMBD Postgresql 8.3 Java Runtime Environment 1.6

78 Dirección General de Servicios de Cómputo Académico / DSSI / Publicaciones Digitales//Octubre 2009 78 Prototipo del proyecto http://dephnis.dgsca.unam.mx/harvester/index.php/in dex Repositorios Información básica de los repositorios Buscador personalizado Menú de Inicio

79 Dirección General de Servicios de Cómputo Académico / DSSI / Publicaciones Digitales//Octubre 2009 79 Sitios Cosechados Más de 56, 000 registros de 8 diferentes repositorios (España) (Brasil) (España) (México)

80 Dirección General de Servicios de Cómputo Académico / DSSI / Publicaciones Digitales//Octubre 2009 80 Menús de la pantalla principal –Inicio, se muestra la página distintiva del sitioInicio, se muestra la página distintiva del sitio –Acerca de, muestra la información básica del sitio –Acceso Abierto, informa sobre el URL base para quePortal de Portales, Latindex pueda ser cosechado –Búsqueda, herramienta que ayudará a los usuarios del sistema a localizar los artículos a texto completo –Idiomas, aparecerán por defecto Español, Ingles y Portugués (hay que trabajar los textos en dichos idiomas) –Ayuda, FAQs, Contacto, son menús cuyos textos hay que definir.

81 Dirección General de Servicios de Cómputo Académico / DSSI / Publicaciones Digitales//Octubre 2009 81 ¿Cómo buscar información en Portal de Portales, Latindex ? Por Repositorio En base de datos del portal Fuera del portal

82 Dirección General de Servicios de Cómputo Académico / DSSI / Publicaciones Digitales//Octubre 2009 82 ¿Cómo buscar información en Portal de Portales, Latindex Búsqueda libre: Resultados de los artículos con la palabra Ciencia Ciencia

83 Dirección General de Servicios de Cómputo Académico / DSSI / Publicaciones Digitales//Octubre 2009 83 ¿Cómo buscar información en Portal de Portales, Latindex ? Búsqueda avanzada A través de metadatos como: Título, Autor, Editor, Fecha de Publicación, etc..

84 Dirección General de Servicios de Cómputo Académico / DSSI / Publicaciones Digitales//Octubre 2009 84 Metadatos en el servidor de Portal de Portales, Latindex Ficha del registro

85 Caso BASE +/- 38 M registros

86

87

88

89