How the remarkable Loving sisters made Fort Worth their home in the 1870s

Not many 19th century women were fortunate to find love and happiness with not one but two distinguished husbands. In most cases the ladies didn’t live long enough, and the pickings were slim for second husbands. Lilly Loving was married in succession to a U.S. Army general and a pioneer Fort Worth merchant. She was intelligent, educated, and mother of three children.

She was born Lilly Loving in Carroll County, Mississippi, on Dec. 6, 1853. Her maternal grandfather was Col. Fielding Davis, a cousin of Confederate President Jefferson Davis, and Lilly was lifelong friends with Varina Howell Davis, wife of Jefferson Davis. Her parents, Christopher and Mary Davis Loving, were progressive and prosperous enough to send their daughter off to a female academy at Bardstown, Kentucky, to get an education. As an 18-year-old student, she received an award for “exemplary conduct and diligence.”

She had two sisters, Ida and Fielding. In 1874, Lilly came to Fort Worth by stagecoach with Ida and their mother, following Fielding, who had settled here and married David R. Crawford a couple of years earlier. In the early 1870s Fort Worth was practically the end of the world. It was family ties that kept the four women on the Trinity River so far from civilization.

After David Crawford died in 1878, Fielding married “the brilliant young [Fort Worth] lawyer,” Frank W. Ball. In 1876, Lilly married another local man, 35-year-old James J. Byrne. He was just not any beau but a general officer in the Union army during the Civil War.

The third sister, Ida, also married a local man, John W. Turner, a successful mining engineer. After his death, Ida carved out a historical niche for herself as a suffragist and entrepreneur.

James Byrne was mustered out of the Army in 1866. Cited more than once for “meritorious conduct and gallantry,” he had no trouble finding a job after the war. He went to work for the Texas & Pacific Railroad as a surveyor, charged with finding a route across the vast west Texas country. While on the job on Aug. 22, 1880, he was ambushed and mortally wounded by Apache Indians.

One of Byrne’s last acts was writing out his will and a farewell letter to Lilly, addressed to “My own, my beloved, my darling wife.” Both were found on his body. The letter was conveyed to her by Maj. C.K. Fairfax, who owned the Transcontinental Hotel. Friends at Fort Quitman brought Byrne’s body back to Fort Worth, and he was buried in Pioneers Rest Cemetery beneath a large, impressive monument commissioned by Lilly. The monument stands in a nice, fenced-in plot with space for additional burials.

He left a widow and daughter (Louella Grace Byrne), although the story gets a little murky here. His estate included valuable property in both New York and Fort Worth, which by the terms of his will left Lilly fixed for life. She lived another 20 years. But if you visit his grave site in Pioneers Rest Cemetery you won’t see her grave, only his. Why, you may wonder, is his wife not buried beside him? That is because Lilly is buried in Oakwood Cemetery beside her second husband.

Second marriage for Lilly Loving

In 1883, Lilly married Max Elser, a prominent Fort Worth merchant and entrepreneur who had moved here from New York in 1872. Lilly was 30 years old when they married. Elser, too, had lost his first spouse, Inez Harding Elser, in 1879. Lilly and Max had two children, Frank Ball Elser and Maximilian “Max” Elser Jr.

Lilly had an active social and civic life. She was active in the Women’s Christian Temperance Union, which fought to bring prohibition to Texas in the 1880s and got to hobnob with professional actors from New York City. She “chaperoned” parties for their children and their children’s friends. Max and Lilly also traveled extensively, spending two months in Mexico City in 1897 pursuing his business interests. When the family traveled, she had a nurse to help look after the children.

James Byrne’s death and Lilly’s remarriage did not close the book on her first marriage. As a widow, she was in no hurry to file Byrne’s will for probate, waiting until 1887. Four years before that, estranged daughter Louella Grace, living in New York, hired lawyers to sue Lilly. She claimed that her mother had expropriated Gen. Byrne’s estate, valued at $50,000.

The legal details are unclear whether it was truly Louella who sued or someone else claiming to be Gen. Byrne’s daughter by an earlier marriage in cahoots with a third woman who wanted payment for taking care of Byrne’s young daughter for years. Confused yet? So was Lilly. She turned the whole mess over to her brother-in-law, Frank W. Ball (Fielding Loving’s second husband).

Ball’s firm took the case to court, and eventually, county clerk John F. Swayne accepted the will for probate, meaning it was valid. As a result, the court found in Lilly’s favor. It was an ugly affair.

Life at home for the new Mrs. Elser was happy otherwise. She had no financial worries, thanks to her second husband. Max sold his store in 1886 and spent the next eight years as a cashier with City National Bank. He also found time to bring the telegraph and the Fort Worth & Denver Railroad into Fort Worth, making money on both. In his spare time he managed the Fort Worth Opera House and developed oil wells and coal mines in Texas and Mexico.

Max still thought of New York City as home, and in 1896 moved his family back there, where they lived until 1900. In 1900 Max Jr. applied to West Point. In July, he passed his exam for admission to the military academy, which should have been cause for celebration in the family. But the family was dealing with tragedy. Lilly had died on April 15 at their home in New York City. She was just 48 years old and had been sick for a while. Ida and Fielding were at her bedside at the end. The three sisters had remained close over the years through multiple marriages and moves.

Max brought Lilly’s body back to Fort Worth for burial. After a funeral service at St. Andrew’s Episcopal Church attended by some of Fort Worth’s most prestigious families, she was interred in Oakwood Cemetery. When Max Elser died 33 years later, he was interred beside Lilly in Oakwood Cemetery, as were her two sisters when they died.

In death, as in life, the Loving sisters remained as close as could be.

Sadly, the only known picture of any of the three sisters is the obituary photo of Ida Loving Turner, who died in 1947. Lilly and Fielding are relegated to the shadows of history.

Author-historian Richard Selcer is a Fort Worth native and proud graduate of Paschal High and TCU.