Tropical islands often exhibit a singular biodiversity within small areas, making them particularly vulnerable to habitat loss and fragmentation. However, we still lack an integrative understanding of how changes in landscape structure impact the dynamics of these ecosystems. In this study, we address the influence of past and present landscape structure on niche-based (deterministic) and neutral (stochastic) components of tree community dynamics in insular forests undergoing habitat loss and fragmentation. We characterized the current taxonomic and functional (wood and leaf traits) composition of 100 tree communities in a dry forest of New Caledonia that underwent habitat loss and fragmentation over the last decades. We used a mechanistic coalescent-based model to simulate community assembly from the observed species pool under varying parameters of immigration and trait-based environmental filtering. By comparing the composition of simulated and observed communities, we inferred parameter values of environmental filtering and immigration shaping the composition of observed communities (Approximate Bayesian Computation). Then, we used partial regressions to evaluate the relationships between inferred parameter values and landscape metrics (distance to edge, patch area, and habitat amount around communities) derived from either recent or past (65 yrs ago) aerial photographs, while controlling for the effect of soil and topography. We found significant relationships between landscape metrics and both parameters related to environmental filtering and immigration. Environmental filtering was affected by both past and recent landscape structure and were mainly influenced by distance to edge and topography. In contrast, immigration rate was primarily linked to past landscape structure, and was positively correlated to habitat amount surrounding communities. Our results show that landscape structure can influence both niche-based and neutral assembly processes in fragmented landscapes. Recent edges creation can affect environmental filtering, resulting in rapid changes in community composition. Nonetheless, the effects of habitat loss and isolation on neutral migration-drift dynamics may take longer to impact community composition. Recently isolated communities may thus be subject to an extinction debt. Our study highlights the importance to include the history of landscapes into conservation policies to better assess the effects of habitat loss and fragmentation on biodiversity dynamics.