This paper presents a comprehensive Bayesian framework for FANOVA models. We provide guidelines for tuning and practical implementation to improve scalability of learning and prediction. Our model is very flexible and can handle different levels of sparsity across and within decomposition orders, as well as among covariates. This flexibility enables the modeling of complex real-world data while enhancing interpretability. Additionally, it allows our model to unify diverse deterministic and Bayesian non-parametric approaches into a single equation, making comparisons and understanding easier. Importantly, our model represents several deterministic methods in a Bayesian way, enabling uncertainty quantification. This general framework opens up possibilities for new model developments that were previously overlooked. For example, we propose a Dirichlet mixing model that addresses limitations of existing models.