The Battle of Palmito Ranch was arguably the last battle of the Civil War. Interestingly, the Confederates won this battle. 

https://www.history.com/news/6-civil-war-battles-after-appomattox

https://www.tshaonline.org/handbook/entries/palmito-ranch-battle-of