Problem(s) Encountered:

This is a weird one.  There have been internet issues stemming from the summit since late morning.  While it now appears that we have internet to our machines again, all hardware physically connected to our network switches are down.  For example, the following devices are now down:

1.3m dome web camera
1.3m tilt sensor web camera
1.3m ethernet-controlled power switch
1.3m telescope server*
1.3m xmis

2.4m xmis

all phones

Interestingly, we somehow have internet and are able to ping all computers, just not physical devices as mentioned above. Not really sure how to proceed on this one as the day, and week, approach an end...

Solution:

[UPDATE] This happened numerous times over following days.  The problem apparently was caused by a “network loop”, created at the summit.  Scary that our systems can be taken down as easily as by an erroneous network cable connection...

No sooner did I file the trouble report and notify tonight’s observer then everything started working again!  I really do not understand what happened to cause all devices to fail like that.  But once I saw one device come back online, all devices followed suit.One other data point—network activity lights for all ports on both switches (1.3m & 2.4m) were solid green when things were fubar.  With everything back to functional, lights are blinking as expected.