Spark solutions for discovering fuzzy association rules in Big Data
MetadataShow full item record
Big DataFuzzy frequent itemsetFuzzy association rulesSpark
Carlos Fernandez-Basso, M. Dolores Ruiz, Maria J. Martin-Bautista, Spark solutions for discovering fuzzy association rules in Big Data, International Journal of Approximate Reasoning, Volume 137, 2021, Pages 94-112, ISSN 0888-613X, [https://doi.org/10.1016/j.ijar.2021.07.004]
SponsorshipCOPKIT project from the 8th Programme Framework (H2020) research and innovation programme 786687; BIGDATAMED projects B-TIC-145-UGR18 P18-RT-2947
The high computational impact when mining fuzzy association rules grows significantly when managing very large data sets, triggering in many cases a memory overflow error and leading to the experiment failure without its conclusion. It is in these cases when the application of Big Data techniques can help to achieve the experiment completion. Therefore, in this paper several Spark algorithms are proposed to handle with massive fuzzy data and discover interesting association rules. For that, we based on a decomposition of interestingness measures in terms of α-cuts, and we experimentally demonstrate that it is sufficient to consider only 10equidistributed α-cuts in order to mine all significant fuzzy association rules. Additionally, all the proposals are compared and analysed in terms of efficiency and speed up, in several datasets, including a real dataset comprised of sensor measurements from an office building.