[statnet_help] Convergence assessment in small networks/with limited data

David Kretschmer dkretsch at mail.uni-mannheim.de
Fri Oct 2 11:08:15 PDT 2020


Hi Martina,

thank you very much for the extremely helpful reply! I wasn’t aware of the fact that the distribution/density plots do not bear significance for convergence but only for inference, but what you described makes perfect sense. Given that parameter estimates (and standard errors) in my analysis are fed into a meta-regression model rather than used for network-level inference, I suppose that this is less of a problem for my setup.

I do have two brief follow-up questions and would be very grateful if you could provide some insight on these too:

1. Regarding the “fuzzy caterpillar”: Given the few observations for the network statistics, the Markov Chain frequently only jumps between a very small number of discrete values (say deviations from the observed value of -1, 0, 1, and 2). Therefore, the chain frequently does not look like a fuzzy caterpillar. However, I suppose that this also is a direct consequence of the low observed number for the network statistic and not an indicator of bad convergence? I suppose that the criterion for MCMC convergence that the chain explores the parameter space then is still fulfilled, but that the parameter space just is much smaller in these situations?

2. In some cases, the chains only alternate between deviations from the observed statistic of -1 and 1, and either never or only infrequently take on a deviation of 0. (In the density/distribution plot, this is reflected in a u-shaped distribution, with low density at a value of 0 and high density at -1 and 1). Do you have any idea for why this pattern occurs and what this means for convergence?

Thank you,
Best
David


--
David Kretschmer
Universität Mannheim
Mannheimer Zentrum für Europäische Sozialforschung (MZES)
A5, 6
68159 Mannheim
Tel.: +49-621-181-2024


> On 2. Oct 2020, at 19:22, martina morris <morrism at uw.edu> wrote:

>

> Hi David,

>

> There are two dx issues here: convergence and statistical inference. The MCMC plots give some insight into both.

>

> Statistical inference:

>

> The principles here are similar to traditional statistical inference -- if you have a small number of observations (esp. for subgroups), the sampling distributions of your statistics may not approximate a symmetric or normal distribution. This is exacerbated by the lower bound at 0.

>

> The MCMC distribution plots (on the right side of the MCMC dx plot layout) are essentially showing the sampling distribution of the stats under the model. Asymmetries here suggest the statistical inference may be compromised. For small nets, you'll often see sawtooth shaped MCMC dx plots. That is also a function of discrete, integer valued stats that only vary over a small range. Not inherently a problem for statistical inference.

>

> Convergence:

>

> Convergence is best assessed using the MCMC traceplots on the left hand side of the plot layout. There, you're looking for a "fuzzy caterpillar". What you don't want to see is a plot that trends up or down, or one that has strong serial correlation in the estimates (some modest correlation is not a problem).

>

> HTH,

> Martina

>

> On Fri, 2 Oct 2020, David Kretschmer wrote:

>

>> Dear all,

>> I estimate models on relatively small networks (about 20-30 nodes), including dyadic covariates with limited information,

>> e.g. with only one or two actual tie observations for the dyadic covariate in some of the networks.

>> I wonder about convergence analysis in this setup, in particular when considering density plots.

>> The values for the network statistics related to the dyadic covariate produced during ERGM estimation have to be zero or

>> positive; they cannot be negative. Because the number of tie observations in the empirical network is so low (only one or

>> two instances), the deviations from these observations shown in the convergence analysis very frequently cannot produce

>> symmetrical density plots: Because the number predicted in the ERGM estimation is constrained to be zero or higher, the

>> deviations are also constrained in one direction. This is also what I observe when looking at density plots for these

>> networks.

>> What I would like to know is what this implies *substantively* for the analysis of these networks: Should criteria for

>> convergence be relaxed in such settings, i.e., is it also fine for the density plots to be asymmetric? Or does this

>> simply mean that results from these networks should not be interpreted at all?

>> Any help would be greatly appreciated.

>> Best,

>> David

>> --

>> David Kretschmer

>> Universität Mannheim

>> Mannheimer Zentrum für Europäische Sozialforschung (MZES)

>> A5, 6

>> 68159 Mannheim

>> Tel.: +49-621-181-2024

>>

>

> ****************************************************************

> Professor Emerita of Sociology and Statistics

> Box 354322

> University of Washington

> Seattle, WA 98195-4322

>

> Office: (206) 685-3402

> Dept Office: (206) 543-5882, 543-7237

> Fax: (206) 685-7419

>

> morrism at u.washington.edu <mailto:morrism at u.washington.edu>

> http://faculty.washington.edu/morrism/ <http://faculty.washington.edu/morrism/>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman13.u.washington.edu/pipermail/statnet_help/attachments/20201002/4ca3ec3a/attachment.html>


More information about the statnet_help mailing list