[statnet_help] standardized coefficients in ERGMs

[Jing Chen]

A lot of thanks to Carter and Robert for your thoughtful responses! They have been very helpful!

What I am trying to compare is the strengths of a few edge effects, which basically are the associations between pairs of network objects. Carter’s idea of turning the question around is inspiring. Instead of diving deep methodologically into the “effect size” issue, Jaccard index and QAP may be good enough for my purpose, given my reviewers are probably not familiar with network analysis.

Thanks again,
Jing

From: statnet_help <statnet_help-bounces at mailman13.u.washington.edu> On Behalf Of Carter T. Butts
Sent: Friday, June 17, 2022 11:58 AM
To: statnet_help at u.washington.edu
Subject: Re: [statnet_help] standardized coefficients in ERGMs


Hi, Jing -

Robert's email pointed out some of the complexities here, but I think one might even more usefully turn the question around.  What do does one want to mean by an effect size?  By a standardized coefficient?  Once that is decided, one can determine (1) if such a quantity is well-defined, and (2) if so, how it can be calculated.  As Robert observed, many ideas that analysts have about effect sizes and standardization are actually special properties of linear models, and do not generalize to the non-linear world.  Such ideas are of limited use in understanding even logistic regression, much less complex systems with "feedback" among the elements (which is what we ERGMs are representing).  But, on the other hand, there are lots of ways to talk about effects and relative effects that can be useful for comparative purposes.  For instance:

 - Trivially, the ERGM coefficients tell you about how the conditional log odds of an edge vary as a function of the covariates, and the rest of the graph.  This provides an effect that has a consistent meaning across networks, and indeed is about as comparable as anything gets in a nonlinear model.  (The meaning is local, in the sense that the change in log odds is taken relative to a particular state, but that's the price of living in a nonlinear world.)  For quantitative covariate effects, one can pick any scale one wants, so if you wanted to partially "standardize" by scaling the covariates to have unit variance, one could do this.  (I tend to think that this type of practice usually causes more problems than it solves in the long run, but there's nothing illegitimate about it.)

 - Via conditional simulation, one can evaluate e.g. the expected change in one or more target statistics, as a function of change in one (or a combination of) parameters.  This is local with respect to the parameter vector, and may or may not be useful to compare across graphs depending on one's choice of statistics, but it does capture (local) "net" effects due to feedback between statistics, etc.  If one has a concrete substantive question relating to some aspect of network structure, computing such effects may be insightful.  One can, further, partially "standardize" these effects by scaling them by the standard deviation of the target statistics (or a function thereof) at the base parameter vector.  This gives you the expected change in statistics per unit change in parameters (local to the current base value), in units of the standard deviation in those statistics.  As with the above case, whether this is a good idea depends on what you want to know, but it can be helpful in giving you a sense of the extent to which small changes in the parameters are making a large difference in network structure, relative to the variation that you would naturally expect to see in that structure.

 - I'm a fan these days of scenario evaluations, and things like virtual "knock-out" experiments: in the latter case, we compare the expectation of some target statistic (or some other distributional statistic) for particular parameters (e.g., the MLE of a fitted model) with what we get if one or more terms in the model are set to zero.  That is, we reach in and "turn off" the term, and see what it does to the graph (as is done physically in a knock-out experiment, where one might e.g., "turn off" expression of some gene in a mouse and see what effect it has).  This can be useful as a probe to better understand how particular mechanisms are contributing to the overall behavior of the model, and can be used to construct a certain type of "effect size" based on how the target changes when specific effects are removed.  (Whether that is useful depends on what you want to know, of course, but it can be insightful.)  We can of course do knock-down/knock-in/knock-up versions, as well, as well as versions where we modify e.g. covariates rather than parameters - all boil down to trying to understand the implications of model terms on substantive behavior by comparing across hypothetical scenarios (whether of empirically plausible or entirely conceptual nature).

 - In some physical settings, the ERGM parameters have a pretty concrete meaning as effective forces: setting aside things like contributions from the reference measure, a given parameter is the energy per unit change in the statistic, times -1/(kT) where T is the system temperature and k is Boltzmann's constant.  So, what we see is the (additive inverse of the) energetic cost of changing a given statistic by one unit, relative to the size of typical energy fluctuations (i.e., kT).  Admittedly, this is not as immediately helpful for social networks, but there are other settings where T is known, and the adjusted parameters then have a very direct and absolute interpretation.  From this vantage point, our usual coefficients are already standardized, in the sense that they reflect (to gloss it a bit) costs of changing the graph relative to available resources.  I think this can be pushed a bit further even in the social case, but I think it is safe to say that this is still something that is being worked out.  We'll have to see where it leads.

Anyway, those are just a few of the examples of things that folks are doing in this area.  I agree with Robert that we're not going to have some simple, generic recipe for how to think about effects that is ideal in all cases, but that doesn't exist even for linear models.  By turns, we have quite a lot of powerful ways to use and interpret the coefficients that we have, and I think that folks will continue to come up with new ones as the number of applications grows.  What needs to be in the driver's seat, in my view, are the substantive questions.  If folks know and can clearly articulate what they are trying to learn, then they are likely to be able to come up with ways to measure the right quantities.

Hope that is helpful,

-Carter
On 6/16/22 4:43 AM, Jing Chen wrote:
Dear Statnet community,

I am posting this question again – not sure if the previous one was sent successfully.

I am trying to compare the strength of an edge effect across ERGMs (the identical model specification, but the outcome objects are different). I am wondering how do we talk about effect sizes in the ERGM world? Are the “regression” coefficients presented in the ERGM output standardized? If not, is there a way to do so?

Any information would be appreciated. Thank you!

Jing Chen, Ph.D.
Assistant professor
Shanghai Jiao Tong University





_______________________________________________

statnet_help mailing list

statnet_help at u.washington.edu<mailto:statnet_help at u.washington.edu>

https://urldefense.com/v3/__http://mailman13.u.washington.edu/mailman/listinfo/statnet_help__;!!CzAuKJ42GuquVTTmVmPViYEvSg!M_zeRwLHiLMJLk_Nxn_dg32pF8ECLVLcU2Sed2IKAaJFtBJa4hT_afhwHhFydSu0iOdUHi2jERtRUQqE9FnFwOxq$<https://urldefense.com/v3/__http:/mailman13.u.washington.edu/mailman/listinfo/statnet_help__;!!CzAuKJ42GuquVTTmVmPViYEvSg!M_zeRwLHiLMJLk_Nxn_dg32pF8ECLVLcU2Sed2IKAaJFtBJa4hT_afhwHhFydSu0iOdUHi2jERtRUQqE9FnFwOxq$>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman13.u.washington.edu/pipermail/statnet_help/attachments/20220617/ae0784a8/attachment.html>