Generating graph-structured data is a challenging task that needs to capture complex relationship between nodes and edges with permutation-invariance property. This paper purposes to generate graphs using latent space energy-based models (EBM). We formulate the latent space EBM as an informative prior distribution, which is trained jointly with the graph generator model. And short-run Markov chain Monte Carlo (MCMC) is employed for the posterior inference and prior sampling process. To further capture the complex distribution of graph data, we provide another version that replaces the EBM prior with a sequence of EBMs, transforming the prior process with a diffusion process. Different graph datasets including generic graphs and molecular graphs are used to validate the method, demonstrating the effectiveness of the latent space EBM and the latent diffusion method.