Tabular models often suffer from distribution shifts due to the complex deployment environments, resulting in performance drop on the target test data. Despite recent advancements in test-time adaptation for vision and language domains, the exploration of test-time adaptation on tabular data (TabTTA) remains limited. TabTTA addresses the task of adapting source models to shifted, unlabeled target domains without access to source data. Existing adaptation methods struggle to handle the heterogeneity and complex dependencies inherent in tabular data, and lack clear guidelines on what knowledge can be reliably transferred across domains. To address this, we propose TabLog, a framework that leverages first-order logic rules to capture and transfer knowledge across tabular domains. TabLog discretizes numerical features, models column-wise dependencies, and introduces a bin-informed contrastive loss for effective test-time training. Experimental results demonstrate TabLog's significant improvement in adaptation performance, accompanied by a comprehensive analysis of the framework and learned logic rules.