According to the FDA:

Potential for Withdrawal Reactions in Sedative/Hypnotic/Anxiolytic Drug- Dependent Patients

Because BuSpar does not exhibit cross-tolerance with benzodiazepines and other common sedative/hypnotic drugs, it will not block the withdrawal syndrome often seen with cessation of therapy with these drugs. Therefore, before starting therapy with BuSpar, it is advisable to withdraw patients gradually, especially patients who have been using a CNS-depressant drug chronically, from their prior treatment. Rebound or withdrawal symptoms may occur over varying time periods, depending in part on the type of drug, and its effective half-life of elimination.

The syndrome of withdrawal from sedative/hypnotic/anxiolytic drugs can appear as any combination of irritability, anxiety, agitation, insomnia, tremor, abdominal cramps, muscle cramps, vomiting, sweating, flu-like symptoms without fever, and occasionally, even as seizures.

Possible Concerns Related to Buspirone's Binding to Dopamine Receptors

Because buspirone can bind to central dopamine receptors, a question has been raised about its potential to cause acute and chronic changes in dopamine-mediated neurological function (eg, dystonia, pseudo-parkinsonism, akathisia, and tardive dyskinesia). Clinical experience in controlled trials has failed to identify any significant neuroleptic-like activity; however, a syndrome of restlessness, appearing shortly after initiation of treatment, has been reported in some small fraction of buspirone-treated patients. The syndrome may be explained in several ways. For example, buspirone may increase central noradrenergic activity; alternatively, the effect may be attributable to dopaminergic effects (ie, represent akathisia). See ADVERSE REACTIONS: Postmarketing Experience.

Physical and Psychological Dependence

In human and animal studies, buspirone has shown no potential for abuse or diversion and there is no evidence that it causes tolerance, or either physical or psychological dependence. Human volunteers with a history of recreational drug or alcohol usage were studied in two double-blind clinical investigations. None of the subjects were able to distinguish between BuSpar and placebo. By contrast, subjects showed a statistically significant preference for methaqualone and diazepam. Studies in monkeys, mice, and rats have indicated that buspirone lacks potential for abuse.

Following chronic administration in the rat, abrupt withdrawal of buspirone did not result in the loss of body weight commonly observed with substances that cause physical dependency.

Although there is no direct evidence that BuSpar causes physical dependence or drug-seeking behavior, it is difficult to predict from experiments the extent to which a CNS-active drug will be misused, diverted, and/or abused once marketed. Consequently, physicians should carefully evaluate patients for a history of drug abuse and follow such patients closely, observing them for signs of BuSpar misuse or abuse (eg, development of tolerance, incrementation of dose, drug-seeking behavior).

References

https://www.accessdata.fda.gov/drugsatfda_docs/label/2010/018731s051lbl.pdf 

https://www.pdr.net/drug-summary/Buspirone-Hydrochloride-Tablets--USP--5-mg--10-mg--15-mg--30-mg--buspirone-hydrochloride-1524

https://www.ncbi.nlm.nih.gov/books/NBK531477/