Reinforced concrete frames with masonry infill walls form the main lateral force-resisting system in a large number of buildings across the world. Furthermore, the geometrical and material parameters of the frame and infill and their interaction affect the lateral force-displacement response of infill wall RC frames. This interaction is affected by the gap between columns and masonry infill. Although limited numerical and experimental research has been done to determine the behavior of the buildings incorporating a masonry wall and an RC frame, little research has been done on the effects of the gap distance between them. However, a systematic investigation into the interaction between the infill wall and the column is more important than ever. Therefore, the current study aims to examine the response of the interaction between the RC frame and the infill walls. A finite element model of a masonry-infilled RC frame was developed with and without gaps, and the influence of column–infill gap distances on the lateral force-displacement response was investigated. The cases studied have been categorized into four modeling types, bare frame model [BF], infill wall model with full contact model [INFC], infill wall with a gap [ING] model, and infill wall with rubber isolator [INRI]. The measured responses include lateral displacement, story drift, story shear force, and overturning moment over building height. Only the effect of lateral load in-planes with constraints in the other directions is investigated in this study. In conclusion, the rubber isolator has the potential to significantly reduce the intensity of earthquake damage; in other words, rubber isolation contributes to improving the structural behavior against seismic loading.