The town In its earliest days consisted of a few houses made of logs or imported lumber. Heart pine shipped from Louisiana was frequently used. Brick buildings were scarce but slaves made bricks for foundations and chimneys. On the riverbank where the City Hall now stands was Lucinda Flowers' cabin, built about 1843, and nearby were cabins for about one hundred slaves. Farther away on the main road, towards Matagorda was the two-story brick-and-scantlings plantation home of Albert Horton, elected Lieutenant Governor in 1845. A. A. McWllie's 228 acre plantation was located at the northern edge of town.
The first courthouse of Wharton County, a small frame building, was built in 1848, enclosed within a wire fence in 1851. In June, 1852, the first courthouse was torn down. The new courthouse was finished in March of 1852. Built of brick, the 401 square building was two stories, with the first floor divided by intersecting north-south and east-west passageways. In 1857, the wire fence was replaced by a plank fence of heart pine which stood until the early 1900's.
The second official building to be built was the jail in 1854. It was made of 811 logs on a 31 brick base, topped by a second story made of a wooden wall enclosed by a brick wall