A New Knowledge Capitalization Framework in the Big Data Context Through Shared Parameters Experiences.

Knowledge management proves to be inexorable in generating value from disorganized knowledge bases, as well as separating concerns through intelligent knowledge capitalization system in the big data context. Such systems, however, require a long and challenging learning process and complex parameters tuning in order to push the capitalization process forward. In this paper, a new knowledge capitalization framework is introduced as an adaptive and intelligent technique, acting on top of a distributed system and running on a large scale. This framework is a three-level paradigm in which each knowledge base is modeled as a mixture over an underlying set of knowledge groups. Each group is, in turn, formed as a mixture over a latent set of knowledge entities. Besides, focusing on each model separately and tuning its parameters require more extended time and resources to find the optimal configuration, so the proposed approach uses the shared parameter mechanism driven by the group coherence metric. It relies on this paradigm to increase the model’s quality, improve knowledge entities’ coherence, and advance the groups’ smoothness and density. Results reveal significant and robust consistency amongst different knowledge groups. Additionally, each distributed model is updated three times on average. A straightforward adaptation of each model can lead to an improved model, with an augmentation of $$20\%$$ in the group coherence. Finally, a knowledge retrieval system is developed to verify the appropriateness and efficacy of the formed groups as well as to evaluate the response time and precision.