akips
This akips python module provides a simple way for python scripts to interact with the AKiPS Network Monitoring Software Web API interface.
1""" 2This akips python module provides a simple way for python scripts to interact with 3the AKiPS Network Monitoring Software Web API interface. 4""" 5 6__version__ = "1.4.0" 7 8import csv 9import io 10import logging 11import re 12import warnings 13from datetime import datetime 14from typing import Any, cast 15 16import pytz 17import requests 18import urllib3 19 20from akips.exceptions import ( 21 AkipsAuthenticationError, 22 AkipsCredentialError, 23 AkipsError, 24 AkipsSectionDisabledError, 25) 26 27# Logging configuration 28logger = logging.getLogger(__name__) 29 30 31class AKIPS: 32 """ 33 A class to handle interactions with the AKiPS Web API 34 35 AKiPS ships two API accounts, api-ro and api-rw, and its sections do not 36 all accept the same one. Supply the passwords for whichever accounts you 37 need and each call uses the right one; see SECTION_USERS below for the 38 mapping. A caller only reading data needs ro_password alone. 39 40 api = AKIPS('akips.example.com', ro_password='...', rw_password='...') 41 42 Four of the ten sections have methods of their own here: api-db, 43 api-script, api-msg and api-availability. The rest are reached through 44 call(), which sends a request to any section and parses the reply in the 45 same shapes those methods use. 46 47 AKiPS stores data in three levels, a parent such as a device or user, 48 then a child such as an interface or 'sys', then an attribute. What an 49 attribute's value means depends on its type, per the AKiPS API guide: 50 51 counter always 1, so the value carries nothing 52 enum '{integer},{text}', e.g. '2,down' 53 gauge a scale factor, positive to multiply and negative to 54 divide, not a reading 55 integer a whole number, positive, negative or zero 56 RTT microseconds, not milliseconds 57 text up to 2000 characters 58 timestamp seconds since the Unix epoch 59 uptime seconds since the status last changed 60 61 Counters and gauges therefore come back from get_attributes() as their 62 definition rather than a reading; the readings are in the time series 63 database, which get_latest_values() and get_series() read. 64 65 Time filters are not all the same shape either. 'lastNm' and 'lastNh' 66 are rolling windows, while 'lastNd' is calendar relative, so 'last1d' is 67 today rather than 24 hours. AKiPS will say which it means, since 'tf' is 68 one of the commands the read only account can run: 69 70 api.call('tf span last24h') 71 api.call('tf dump last1d') 72 73 """ 74 75 # Every API section AKiPS publishes, mapped to the account it accepts, as 76 # documented on the server's own Web API settings page. Every section 77 # takes api-ro except api-script, which requires api-rw, and api-db, which 78 # takes either: api-ro for read-only commands and api-rw for all of them. 79 # None marks that pair, where the read only account is preferred and a 80 # command needing more rights is reached with user='rw'. 81 # 82 # Only api-db reads a username at all. The others authenticate on the 83 # password alone, so the username sent alongside is ignored there. 84 # 85 # This doubles as the list of sections known to exist. Calling one that 86 # is not here is warned about rather than refused, because AKiPS may add 87 # sections and waiting for a release here would defeat the point of 88 # call(). Each section is also disabled by default on the server, so a 89 # section listed here can still be rejected until it is enabled. 90 SECTION_USERS: dict[str, str | None] = { 91 "api-availability": "api-ro", 92 "api-config-viewer": "api-ro", 93 "api-db": None, 94 "api-flow": "api-ro", 95 "api-flow-timeseries": "api-ro", 96 "api-http-log": "api-ro", 97 "api-msg": "api-ro", 98 "api-script": "api-rw", 99 "api-spm": "api-ro", 100 "api-unused-interfaces": "api-ro", 101 } 102 """Every API section AKiPS publishes, mapped to the account it accepts. 103 None marks a section taking either, where the read only account is 104 preferred. Also the list of sections known to exist.""" 105 106 SECTION_METHODS: dict[str, str] = { 107 "api-script": "GET", 108 } 109 """HTTP method to use per section, for the sections that cannot take the 110 default. Anything absent here is sent as POST when use_post is on. 111 112 **api-script does not answer a POST.** The server returns 200 headers in 113 about a quarter of a second, then sends no body and holds the connection 114 open until the client gives up, so every site script call hangs. The same 115 call as GET returns normally, and api-db takes a POST with the identical 116 header, so it is api-script specifically. Reported to AKiPS 2026-08-21. 117 118 **The cost is that those calls put the password back in the query 119 string**, which is what use_post exists to prevent. It applies to 120 get_device_by_ip(), set_group_membership() and delete_device(), and it is 121 api-rw for two of them. GET is not a preference: it is what the section 122 answers, and there is no third option, since the alternative is a call 123 that never returns. 124 125 Nothing here assumes that will change. Sending the password in a POST 126 body is itself undocumented — AKiPS support gave it out rather than the 127 API guide describing it — so what any given server accepts is a question 128 for that server rather than something this module can predict. This is a 129 class attribute for that reason: if a server does take a POST on a 130 section, say so without waiting for a release here. 131 132 AKIPS.SECTION_METHODS["api-script"] = "POST" 133 134 That is class wide and affects every client in the process.""" 135 136 def __init__( 137 self, 138 server: str, 139 username: str = "api-ro", 140 password: str | None = None, 141 verify: bool | str = True, 142 timezone: str = "America/New_York", 143 timeout: int = 30, 144 ro_password: str | None = None, 145 rw_password: str | None = None, 146 use_post: bool = True, 147 ) -> None: 148 self.server = server 149 """The AKiPS server hostname or IP address.""" 150 self.username = username 151 """Kept for callers who set it directly. With 'api-ro' or 'api-rw' 152 the password given alongside fills that account; with any other name 153 that pair is used for every section, which is how to use a custom 154 AKiPS API account.""" 155 self.password = password 156 """The password paired with username.""" 157 self.ro_password = ro_password 158 """Password for the api-ro account.""" 159 self.rw_password = rw_password 160 """Password for the api-rw account.""" 161 self.verify = verify 162 """Whether to verify TLS certificates. A path to a CA bundle can be 163 given instead, which is how to trust a server whose chain is missing 164 an intermediate without turning verification off entirely.""" 165 self.server_timezone = timezone 166 """Timezone of the AKiPS server, used to read the epochs it sends.""" 167 self.timeout = timeout 168 """HTTP timeout in seconds applied to every call. Assign to it to 169 change the timeout of an existing client, e.g. api.timeout = 60.""" 170 self.use_post = use_post 171 """Whether to send the password in a POST body instead of the query 172 string. True by default, and it should stay that way: URLs are 173 recorded by web servers, proxies and load balancers in their access 174 logs, and appear in exception messages and client history. A request 175 body is not logged that way, and a credential does not belong in a URL. 176 177 Set it to False only for a server that will not accept the POST form, 178 which puts the password back in the URL. Nothing falls back on its 179 own, because a silent retry over GET would leak the password at 180 exactly the moment the server turned out not to support this.""" 181 self.session = requests.Session() 182 """The requests session every call is made through.""" 183 # Sections warned about already, so a caller legitimately using a 184 # section this release does not know about is told once rather 185 # than on every call 186 self._unknown_sections: set[str] = set() 187 # Sections already warned about for falling back to GET, so a poll 188 # loop is told once rather than on every call 189 self._method_warned: set[str] = set() 190 191 # A username other than the two built in accounts is used for every 192 # section. AKiPS does not offer custom API accounts yet, but this is 193 # where they will land, and it keeps working for anyone already 194 # passing username and password directly. 195 self._account_override: tuple[str, str] | None = None 196 if password is not None: 197 if username == "api-ro" and self.ro_password is None: 198 self.ro_password = password 199 elif username == "api-rw" and self.rw_password is None: 200 self.rw_password = password 201 elif username not in ("api-ro", "api-rw"): 202 self._account_override = (username, password) 203 204 if ( 205 self._account_override is None 206 and self.ro_password is None 207 and self.rw_password is None 208 ): 209 raise AkipsCredentialError( 210 "No AKiPS password provided. Pass ro_password, rw_password, " 211 "or a username and password pair." 212 ) 213 214 # --------------------------------------------------------------------------- 215 # api-db interface methods, these use the 'api-ro' or 'api-rw' user 216 217 # entities commands 218 219 def get_devices( 220 self, group_filter: str = "any", groups: list[str] | None = None 221 ) -> dict[str, dict[str, str | None]] | None: 222 """ 223 Pull a list of all devices and six key attributes of each, optionally 224 filtered by group membership. 225 226 This reads the 'sys' child and nothing else, and asks it for six 227 attributes: ip4addr, SNMPv2-MIB.sysName, SNMPv2-MIB.sysDescr, 228 SNMPv2-MIB.sysObjectID, SNMPv2-MIB.sysLocation and 229 SNMPv2-MIB.sysContact. Those are the values the AKiPS device edit page 230 shows read only, being what SNMP reported rather than what an operator 231 set, plus the address. 232 233 Both the child and the six are fixed here rather than arguments, 234 because this is the inventory view: every device comes back carrying 235 all six, as None where it reported no value, so they can be listed or 236 tabulated without checking each key first. Anything else the server 237 returns for a device is kept alongside them rather than dropped. 238 239 sysObjectID is worth knowing about: it identifies the model, such as 240 'ARUBA-MIB.ap225', which is often the field an inventory actually wants 241 and is more reliably populated than sysLocation. 242 243 For other attributes, other children, or a device's whole contents, 244 see get_attributes() and get_device(). 245 246 Because it asks for one child, this is the only method returning 247 attributes that does not keep the child level; the result is flattened 248 to device and attribute, which is the shape a listing wants. Should a 249 reply ever carry more than one child, their attributes are merged and 250 the last one read wins, where get_attributes() would keep them apart. 251 252 Supporting AKiPS command syntax: 253 254 mget {type} [{parent regex} [{child regex} [{attribute regex}]]] 255 [descr {/regex/}] [value {text|integer|/regex/}] 256 [profile {profile name}] [any|all|not group {group name} ...] 257 258 Args: 259 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 260 groups (list): list of group names to filter by (if any) 261 Returns: 262 A dictionary of device names to attribute dictionaries, or None if no devices found 263 Raises: 264 AkipsError: if the AKiPS server returns an error 265 """ 266 # The polled values the AKiPS device edit page shows read only, plus 267 # the address. Keeping to that set is deliberate: they are the 268 # standard SNMP system group fields an operator already recognizes. 269 attributes = [ 270 "ip4addr", 271 "SNMPv2-MIB.sysName", 272 "SNMPv2-MIB.sysDescr", 273 "SNMPv2-MIB.sysObjectID", 274 "SNMPv2-MIB.sysLocation", 275 "SNMPv2-MIB.sysContact", 276 ] 277 cmd_attributes = "|".join(attributes) 278 params = { 279 "cmds": f"mget text * sys /{cmd_attributes}/", 280 } 281 if groups: 282 # [any|all|not group {group name} ...] 283 group_list = " ".join(groups) 284 params["cmds"] += f" {group_filter} group {group_list}" 285 text = self._get(params=params) 286 if text: 287 data: dict[str, dict[str, str | None]] = {} 288 for parent, children in self._parse_attributes(text).items(): 289 # Every requested attribute is present, as None where the 290 # device reported no value for it 291 entry: dict[str, str | None] = dict.fromkeys(attributes) 292 for child_attributes in children.values(): 293 entry.update(child_attributes) 294 data[parent] = entry 295 # A reply that parses to nothing is nothing found, the same 296 # answer an empty reply gives, rather than an empty container 297 if not data: 298 return None 299 logger.debug("Found {} devices in akips".format(len(data.keys()))) 300 return data 301 return None 302 303 def get_device(self, device: str) -> dict[str, dict[str, str | None]] | None: 304 """ 305 Pull all configuration attributes for a single device. The name is the 306 device's AKiPS name, its one primary key, which is either its sysName 307 or its IP address depending on how the server is set to name devices. 308 It is assigned at discovery, but a server can be told to reassign 309 devices already discovered from the other source, and an operator can 310 change one by hand, so a caller storing these as identifiers of its own 311 should not assume they never change. A device keyed by name still 312 carries its address as an attribute, and get_device_by_ip() resolves 313 an address back to the key. 314 315 This is the deep dive: every child and attribute this device holds, 316 which varies by device type. For the same fields across every device, 317 see get_devices(). 318 319 The result is the one device's children and their attributes, not a 320 dictionary keyed by the name that was just passed in: 321 322 device = api.get_device('TH840-A') 323 device['sys']['ip4addr'] 324 325 get_attributes() is the same query without that assumption, and keys 326 its result by device because it can match several. 327 328 Supporting AKiPS command syntax: 329 330 mget {type} [{parent regex} [{child regex} [{attribute regex}]]] 331 [descr {/regex/}] [value {text|integer|/regex/}] 332 [profile {profile name}] [any|all|not group {group name} ...] 333 334 Args: 335 device (str): the AKiPS name of one device, exactly. AKiPS matches 336 a bare name in this position exactly, so this is unambiguous; a 337 '/regex/' is refused, since matching several devices is what 338 get_attributes() is for 339 Returns: 340 A dictionary of the device's child names to attribute names and 341 values, or None if the device was not found 342 Raises: 343 ValueError: if a pattern is given, or if the reply somehow held 344 more than one device 345 AkipsError: if the AKiPS server returns an error 346 """ 347 # A bare name matches exactly in the parent position, but a '/regex/' 348 # does not, and this method has nowhere to put a second device. 349 if device.startswith("/") and device.endswith("/") and len(device) > 1: 350 raise ValueError( 351 "get_device takes one device name, not a pattern. Use " 352 "get_attributes(device={!r}) to match several".format(device) 353 ) 354 # get_attributes() with the filters left at their defaults, rather than 355 # building the same command a second time. 356 data = self.get_attributes(device=device) 357 if not data: 358 return None 359 if len(data) > 1: 360 raise ValueError( 361 "get_device matched {} devices ({}). Use get_attributes() " 362 "for more than one".format(len(data), ", ".join(sorted(data))) 363 ) 364 return next(iter(data.values())) 365 366 # The children ping and SNMP state are reported under. Naming them saves 367 # AKiPS walking every child of every device, which is most of the cost of 368 # this query: on a 16,000 device fleet the wildcard took 10.5s against 369 # 5.0s here, for the same rows. ping6 is listed though most sites monitor 370 # over IPv4 alone, because a device reachable only over IPv6 going down is 371 # exactly what this call must not miss, and an alternative that matches 372 # nothing costs nothing. 373 UNREACHABLE_CHILDREN = "ping4|ping6|sys" 374 """The children get_unreachable() searches, as a regex.""" 375 376 def get_unreachable( 377 self, children: str = UNREACHABLE_CHILDREN 378 ) -> dict[str, dict[str, Any]] | None: 379 """ 380 Pull a list of unreachable devices by Ping and SNMP state. 381 382 Supporting AKiPS command syntax: 383 384 mget {type} [{parent regex} [{child regex} [{attribute regex}]]] 385 [descr {/regex/}] [value {text|integer|/regex/}] 386 [profile {profile name}] [any|all|not group {group name} ...] 387 388 Note the {type} field is left off here. These are enum attributes, so 389 narrowing with 'mget text' returns no rows at all rather than an 390 error, even though it is the obvious thing to reach for. 391 392 **Asking only for what is broken is not necessarily the cheap way.** 393 On one fleet this call took roughly three times as long as 394 get_ping_state(), while returning 289 times fewer rows: filtering on 395 value makes AKiPS match every device's enum rather than dump the 396 attribute, and this asks for two attributes where that asks for one. 397 Whether that holds elsewhere is unknown, and it may be a property of 398 that server's data rather than of the query. It is recorded because 399 the opposite is the natural assumption: if this call is hot, measure 400 it against reading the state outright and filtering here. 401 402 Args: 403 children (str): regex of children to search, defaulting to the 404 ones AKiPS reports these attributes under. Pass '*' for 405 every child of every device, which is correct for a site 406 naming them differently and considerably slower 407 Returns: 408 A dictionary of device names to their unreachable attributes, or 409 None if nothing was reported as down 410 Raises: 411 AkipsError: if the AKiPS server returns an error 412 """ 413 # '*' is the wildcard rather than a pattern, so it is the one value 414 # that must not be wrapped in slashes 415 child = children if children == "*" else f"/{children}/" 416 params = { 417 "cmds": f"mget * * {child} /PING.icmpState|SNMP.snmpState/ value /down/", 418 } 419 text = self._get(params=params) 420 if text: 421 data: dict[str, dict[str, Any]] = {} 422 unparsed = [] 423 lines = text.split("\n") 424 for line in lines: 425 match = re.match( 426 r"^(\S+)\s(\S+)\s(\S+)\s=\s(\S+),(\S+),(\S+),(\S+),(\S+)?$", line 427 ) 428 if not match: 429 if line.strip(): 430 # A line reporting a device down that this does not 431 # understand must not vanish: under reporting an 432 # outage is the worst thing this call can do. 433 unparsed.append(line) 434 continue 435 # epoch fields are in the server's timezone 436 name = match.group(1) 437 attribute = match.group(3) 438 event_start = datetime.fromtimestamp( 439 int(match.group(7)), tz=pytz.timezone(self.server_timezone) 440 ) 441 device_added = datetime.fromtimestamp( 442 int(match.group(6)), tz=pytz.timezone(self.server_timezone) 443 ) 444 if name not in data: 445 # populate a starting point for this device 446 data[name] = { 447 "name": name, 448 "ping_state": "n/a", 449 "snmp_state": "n/a", 450 "event_start": event_start, # epoch in local timezone 451 } 452 if attribute == "PING.icmpState": 453 data[name]["ping_state"] = match.group(5) 454 # A device down on both checks reports one child, index 455 # and address. Ping wins them, because it is the only 456 # line carrying an address, and assigning here while the 457 # SNMP branch below only fills gaps makes the result the 458 # same whichever order the lines arrive in. 459 data[name]["child"] = match.group(2) 460 data[name]["index"] = match.group(4) 461 data[name]["device_added"] = device_added 462 data[name]["ip4addr"] = match.group(8) 463 elif attribute == "SNMP.snmpState": 464 data[name]["snmp_state"] = match.group(5) 465 data[name].setdefault("child", match.group(2)) 466 data[name].setdefault("index", match.group(4)) 467 data[name].setdefault("device_added", device_added) 468 data[name].setdefault("ip4addr", None) 469 # A device down on both ping and SNMP has two start times; the 470 # outage began at the earlier of them. This has to be the only 471 # place event_start is set, or the comparison is against the 472 # value just written from this same line and the last line seen 473 # would always win. 474 if event_start < data[name]["event_start"]: 475 data[name]["event_start"] = event_start 476 if unparsed: 477 logger.warning( 478 "Could not parse {} of {} unreachable lines from akips, " 479 "those devices are missing from the result. First: {}".format( 480 len(unparsed), len(unparsed) + len(data), unparsed[0][:200] 481 ) 482 ) 483 # A reply that parses to nothing is nothing found, the same 484 # answer an empty reply gives, rather than an empty container 485 if not data: 486 return None 487 logger.debug("Found {} devices in akips".format(len(data))) 488 return data 489 return None 490 491 def get_ping_state( 492 self, 493 states: tuple[str, ...] | list[str] | None = None, 494 child: str = "ping4", 495 group_filter: str = "any", 496 groups: list[str] | None = None, 497 ) -> dict[str, dict[str, Any]] | None: 498 """ 499 Pull the ping state of every device, with when it last changed. 500 501 get_unreachable() answers 'what is broken now' and so asks only for 502 the devices that are down. This asks the same record without that 503 filter, which is what to call for a device that is up: its state, and 504 both of the epochs the enum carries. 505 506 **The two epochs are the useful part and their names undersell them.** 507 'created' is when AKiPS started polling the device, so it is the date 508 the device was added to AKiPS, and it is not otherwise reachable for a 509 device that is healthy. 'modified' is the instant the state last 510 changed, not a row-touched timestamp. Both arrive as aware datetimes 511 in the server's timezone rather than as the integers the raw attribute 512 holds, so nothing needs converting. 513 514 Those two answer the column AKiPS shows on its own device dashboard, 515 the one reading Uptime on a device that is up and Downtime on a device 516 that is down. It is not sysUpTime: the figure is now minus 'modified' 517 and 'value' decides which word. sysUpTime counts from the last boot 518 and keeps counting through an outage, so the two disagree on exactly 519 the devices somebody is looking at. 520 521 get_snmp_state() is the same record for the SNMP agent. It answers 522 for far fewer devices, because AKiPS pings everything it holds and 523 polls SNMP only where SNMP is configured. 524 525 Args: 526 states (list): only report these states, e.g. ('down',), or None 527 for every device whatever its state, which is the default 528 child (str): which ping child to read (default: 'ping4'). Pass 529 'ping4|ping6' for both, though a device answering on both 530 keeps only one entry and warns, since this is keyed by device 531 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 532 groups (list): list of group names to filter by (if any) 533 Returns: 534 A dictionary of device names to the parsed state, or None if no 535 device matched. Each entry carries the enum fields described on 536 _parse_enum, plus the device 'name' and 'child' 537 Raises: 538 AkipsError: if the AKiPS server returns an error 539 """ 540 return self._get_enum_attribute( 541 "PING.icmpState", 542 child=child, 543 values=states, 544 group_filter=group_filter, 545 groups=groups, 546 ) 547 548 def get_snmp_state( 549 self, 550 states: tuple[str, ...] | list[str] | None = None, 551 child: str = "sys", 552 group_filter: str = "any", 553 groups: list[str] | None = None, 554 ) -> dict[str, dict[str, Any]] | None: 555 """ 556 Pull the SNMP agent state of every SNMP polled device. 557 558 The SNMP counterpart to get_ping_state(), reading the same kind of 559 record with the same fields: the state, when the device was added, 560 and when the state last changed. 561 562 **It answers for fewer devices than get_ping_state() does, and the 563 difference is large.** AKiPS pings everything it holds but polls SNMP 564 only where SNMP is configured, so a device absent from this result is 565 usually one that is not SNMP polled rather than one whose agent has 566 stopped answering. Measured on one fleet, 5,821 devices of 16,785 567 appeared here and the rest were ICMP only. A caller that assumes the 568 same denominator as the ping call reads two thirds of the fleet as 569 broken. 570 571 Absence therefore means unknown, not down. The devices answering here 572 are the same ones answering SNMPv2-MIB.sysUpTime, which is a way to 573 confirm the denominator on a given server. 574 575 Args: 576 states (list): only report these states, e.g. ('down',), or None 577 for every SNMP polled device, which is the default 578 child (str): which child holds the agent state (default: 'sys') 579 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 580 groups (list): list of group names to filter by (if any) 581 Returns: 582 A dictionary of device names to the parsed state, or None if no 583 device matched. Each entry carries the enum fields described on 584 _parse_enum, plus the device 'name' and 'child' 585 Raises: 586 AkipsError: if the AKiPS server returns an error 587 """ 588 return self._get_enum_attribute( 589 "SNMP.snmpState", 590 child=child, 591 values=states, 592 group_filter=group_filter, 593 groups=groups, 594 ) 595 596 def get_attributes( 597 self, 598 device: str = "*", 599 child: str = "*", 600 attribute: str = "*", 601 value: str | None = None, 602 group_filter: str = "any", 603 groups: list[str] | None = None, 604 ) -> dict[str, dict[str, dict[str, str | None]]] | None: 605 """ 606 Pull attribute values with variable search criteria. Search criteria defaults to 607 a wildcard match but can be filtered by 'device' name or pattern, 'child' name or pattern, 608 'attribute' name or pattern, and/or attribute 'value' or pattern. Additionally, 609 results can be filtered by group membership using 'any', 'all', or 'not' operators 610 along with one or more group names. 611 612 Supporting AKiPS command syntax: 613 614 mget {type} [{parent regex} [{child regex} [{attribute regex}]]] 615 [descr {/regex/}] [value {text|integer|/regex/}] 616 [profile {profile name}] [any|all|not group {group name} ...] 617 618 **A bare attribute name matches exactly and matches nothing.** AKiPS 619 qualifies most attributes with their MIB, so 'sysUpTime' matches no 620 attribute anywhere while 'SNMPv2-MIB.sysUpTime' matches on every 621 device that reports it. The unqualified form is not an error: it 622 returns an empty result, which looks exactly like a fleet where 623 nothing reports that attribute. Use a pattern, '/sysUpTime/', when 624 the qualified name is not known. 625 626 Args: 627 device (str): device name or pattern to match (default: '*') 628 child (str): child name or pattern to match (default: '*') 629 attribute (str): attribute name or pattern to match (default: '*'). 630 A bare name must match exactly, MIB prefix included; see above 631 value (str): value or pattern to match (default: None) 632 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 633 groups (list): list of group names to filter by (if any) 634 Returns: 635 A nested dictionary of device names to child names to attribute names and values, 636 or None if no devices found 637 Raises: 638 AkipsError: if the AKiPS server returns an error 639 """ 640 params = { 641 "cmds": f"mget * {device} {child} {attribute}", 642 } 643 if value: 644 # [value {text|/regex/|integer|ipaddr}] 645 params["cmds"] += f" value {value}" 646 if groups: 647 # [any|all|not group {group name} ...] 648 group_list = " ".join(groups) 649 params["cmds"] += f" {group_filter} group {group_list}" 650 text = self._get(params=params) 651 if text: 652 data = self._parse_attributes(text) 653 # A reply that parses to nothing is nothing found, the same 654 # answer an empty reply gives, rather than an empty container 655 if not data: 656 return None 657 logger.debug("Found {} devices in akips".format(len(data.keys()))) 658 return data 659 return None 660 661 # UPS helpers 662 663 # UPS output sources other than 'normal'. A UPS reporting any of these is 664 # not running on mains, which is what an operator wants to know about. 665 # 666 # In MIB order, since UPS-MIB numbers them other(1) none(2) normal(3) 667 # bypass(4) battery(5) booster(6) reducer(7). 'none' is a UPS delivering 668 # no output at all and 'other' one that cannot classify its own source; 669 # both are here for the same reason 'unknown' is in the battery states 670 # below, because a UPS that cannot answer the question is worth looking 671 # at too. 672 UPS_ABNORMAL_OUTPUT_SOURCES = ( 673 "other", 674 "none", 675 "bypass", 676 "battery", 677 "booster", 678 "reducer", 679 ) 680 """Output sources get_ups_output_source() reports by default, being every 681 UPS-MIB source but normal, so every state that is not running on mains.""" 682 683 # Battery states other than batteryNormal. 'unknown' is included because 684 # a UPS that cannot report its own battery is worth looking at too. 685 UPS_ABNORMAL_BATTERY_STATES = ("unknown", "batteryLow", "batteryDepleted") 686 """Battery states get_ups_battery_status() reports by default, being 687 every UPS-MIB state but batteryNormal.""" 688 689 # The attribute Liebert and Vertiv equipment reports battery test results 690 # in. Battery test results are not in the standard UPS-MIB, so every 691 # vendor uses its own; this one is named in the method that reads it. 692 LIEBERT_BATTERY_TEST_ATTRIBUTE = "LIEBERT-GP-POWER-MIB.lgpPwrBatteryTestResult" 693 """The attribute get_liebert_battery_test() reads. Battery test results 694 are not in the standard UPS-MIB, so this one is vendor specific.""" 695 696 def get_ups_battery_status( 697 self, 698 states: tuple[str, ...] | list[str] | None = UPS_ABNORMAL_BATTERY_STATES, 699 group_filter: str = "any", 700 groups: list[str] | None = None, 701 ) -> dict[str, dict[str, Any]] | None: 702 """ 703 Pull the UPS devices whose battery is not reporting as normal. 704 705 UPS-MIB reports the battery's own condition, separately from where the 706 UPS is drawing its output, which get_ups_output_source() reads. By 707 default this returns only the states other than batteryNormal. 708 709 This is the battery's condition, not how long it would last. AKiPS 710 keeps the numeric readings such as upsEstimatedMinutesRemaining in its 711 time series database rather than alongside these, so they come from 712 get_series() rather than from here. Reading them with mget returns 713 the gauge's scaling factor, which is identical for every device. 714 715 Args: 716 states (list): battery states to report, defaulting to everything 717 except batteryNormal. Pass None for every UPS whatever its 718 battery state 719 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 720 groups (list): list of group names to filter by (if any) 721 Returns: 722 A dictionary of device names to the parsed state, or None if no 723 device matched. Each entry carries the enum fields described on 724 _parse_enum, where 'value' is the battery state and 'modified' is 725 when it last changed, plus the device 'name' and 'child' 726 Raises: 727 AkipsError: if the AKiPS server returns an error 728 """ 729 return self._get_enum_attribute( 730 "UPS-MIB.upsBatteryStatus", 731 child="battery", 732 values=states, 733 group_filter=group_filter, 734 groups=groups, 735 ) 736 737 def get_ups_output_source( 738 self, 739 states: tuple[str, ...] | list[str] | None = UPS_ABNORMAL_OUTPUT_SOURCES, 740 group_filter: str = "any", 741 groups: list[str] | None = None, 742 ) -> dict[str, dict[str, Any]] | None: 743 """ 744 Pull the UPS devices that are not running on mains power. 745 746 UPS-MIB reports where a UPS is drawing its output from, which is 747 'normal' when all is well. By default this returns every other value, 748 so the result is the list of UPSes worth looking at. That includes 749 'none', a UPS delivering no output at all, and 'other', one that 750 cannot classify its own source. 751 752 Note this is the output source, not the battery's own health, which 753 UPS-MIB reports separately as upsBatteryStatus. 754 755 Args: 756 states (list): output sources to report, defaulting to everything 757 except 'normal'. Pass None for every UPS whatever its state 758 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 759 groups (list): list of group names to filter by (if any) 760 Returns: 761 A dictionary of device names to the parsed state, or None if no 762 device matched. Each entry carries the enum fields described on 763 _parse_enum, where 'value' is the output source and 'modified' is 764 when it last changed, plus the device 'name' and 'child' 765 Raises: 766 AkipsError: if the AKiPS server returns an error 767 """ 768 return self._get_enum_attribute( 769 "UPS-MIB.upsOutputSource", 770 # The child the UPS itself is reported under, as against 771 # 'battery' for the battery attributes. Naming it keeps AKiPS 772 # from walking every child of every device, which is most of the 773 # cost of the query. 774 child="ups", 775 values=states, 776 group_filter=group_filter, 777 groups=groups, 778 ) 779 780 def get_liebert_battery_test( 781 self, 782 results: tuple[str, ...] | list[str] | None = ("failed",), 783 attribute: str = LIEBERT_BATTERY_TEST_ATTRIBUTE, 784 group_filter: str = "any", 785 groups: list[str] | None = None, 786 ) -> dict[str, dict[str, Any]] | None: 787 """ 788 Pull the results of the last battery self test on Liebert and Vertiv 789 UPS equipment. 790 791 By default this returns only the failures, which is the list of 792 batteries to replace. Pass results=None for every UPS and its last 793 result. 794 795 The vendor is in the name on purpose. Battery test results are not in 796 the standard UPS-MIB, so this reads an attribute only Liebert and 797 Vertiv equipment reports. Run against another vendor's fleet it 798 returns nothing, which would otherwise read as good news. Another 799 vendor's equivalent attribute can be passed to reuse the same parsing 800 and shape. 801 802 Args: 803 results (list): test results to report, defaulting to failures 804 only. Pass None for every UPS whatever its last result 805 attribute (str): the vendor attribute holding the result 806 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 807 groups (list): list of group names to filter by (if any) 808 Returns: 809 A dictionary of device names to the parsed result, or None if no 810 device matched. Each entry carries the enum fields described on 811 _parse_enum, where 'value' is the test result and 'modified' is 812 when it last changed, plus the device 'name' and 'child' 813 Raises: 814 AkipsError: if the AKiPS server returns an error 815 """ 816 return self._get_enum_attribute( 817 attribute, 818 child="battery", 819 values=results, 820 group_filter=group_filter, 821 groups=groups, 822 ) 823 824 # group commands 825 826 def get_group_membership( 827 self, 828 device: str = "*", 829 group_filter: str = "any", 830 groups: list[str] | None = None, 831 ) -> dict[str, list[str]] | None: 832 """ 833 Pull a list of device names to group memberships. Defaults to all devices 834 and all groups (including the special 'maintenance_mode' group). 835 836 Supporting AKiPS command syntax: 837 838 mgroup {type} [{parent regex}] 839 [any|all|not group {group name} ...] 840 841 Args: 842 device (str): device name or pattern to match (default: '*') 843 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 844 groups (list): list of group names to filter by (if any) 845 Returns: 846 A dictionary of device names to lists of group names, or None if no devices found 847 Raises: 848 AkipsError: if the AKiPS server returns an error 849 """ 850 params = { 851 "cmds": f"mgroup * {device}", 852 } 853 if groups: 854 group_list = " ".join(groups) 855 params["cmds"] += f" {group_filter} group {group_list}" 856 text = self._get(params=params) 857 if text: 858 data = { 859 device_name: groups_value.split(",") 860 for device_name, groups_value in self._parse_key_value(text).items() 861 } 862 # A reply that parses to nothing is nothing found, the same 863 # answer an empty reply gives, rather than an empty container 864 if not data: 865 return None 866 logger.debug( 867 "Found {} device and group mappings in akips".format(len(data.keys())) 868 ) 869 return data 870 return None 871 872 # event commands 873 874 def get_events( 875 self, 876 event_type: str = "all", 877 period: str = "last1h", 878 device: str = "*", 879 child: str = "*", 880 attribute: str = "*", 881 group_filter: str = "any", 882 groups: list[str] | None = None, 883 ) -> list[dict[str, str]] | None: 884 """ 885 Pull a list of events over a time period with optional filtering by device, 886 child, attribute, and/or group membership. Defaults to all event types over 887 the last hour. Review AKiPS documentation for details on event types and 888 time filter syntax. 889 890 Supporting AKiPS command syntax: 891 892 mget event {all,critical,enum,threshold,uptime} 893 time {time filter} [{parent regex} {child regex} 894 {attribute regex}] [profile {profile name}] 895 [any|all|not group {group name} ...] 896 897 Args: 898 event_type (str): type of events to retrieve (default: 'all') 899 period (str): time period to retrieve events from (default: 'last1h') 900 device (str): device name or pattern to match (default: '*') 901 child (str): child name or pattern to match (default: '*') 902 attribute (str): attribute name or pattern to match (default: '*') 903 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 904 groups (list): list of group names to filter by (if any) 905 Returns: 906 A list of event dictionaries, or None if no events found 907 Raises: 908 AkipsError: if the AKiPS server returns an error 909 """ 910 params = { 911 "cmds": f"mget event {event_type} time {period} {device} {child} {attribute}" 912 } 913 if groups: 914 # [any|all|not group {group name} ...] 915 group_list = " ".join(groups) 916 params["cmds"] += f" {group_filter} group {group_list}" 917 text = self._get(params=params) 918 if text: 919 data = [] 920 lines = text.split("\n") 921 for line in lines: 922 match = re.match( 923 r"^(\S+)\s(\S+)\s(\S+)\s(\S+)\s(\S+)\s(\S+)\s(.*)$", line 924 ) 925 if match: 926 entry = { 927 "epoch": match.group(1), 928 "parent": match.group(2), 929 "child": match.group(3), 930 "attribute": match.group(4), 931 "type": match.group(5), 932 "flags": match.group(6), 933 "details": match.group(7), 934 } 935 data.append(entry) 936 # A reply that parses to nothing is nothing found, the same 937 # answer an empty reply gives, rather than an empty container 938 if not data: 939 return None 940 logger.debug( 941 "Found {} events of type {} in akips".format(len(data), event_type) 942 ) 943 return data 944 return None 945 946 # time series commands 947 948 def get_series( 949 self, 950 period: str = "last1h", 951 time_interval: int = 60, 952 device: str = "*", 953 attribute: str = "*", 954 get_dict: bool = True, 955 group_filter: str = "any", 956 groups: list[str] | None = None, 957 ) -> list[dict[str, str]] | list[list[str]] | None: 958 """ 959 Pull a series of counter values with average values over a time period with optional 960 filtering by device, attribute, and/or group membership. Defaults to all devices 961 and attributes over the last hour with 60 second intervals. Review AKiPS documentation 962 for details on time filter syntax. 963 964 Supporting AKiPS command syntax: 965 966 cseries [interval total|avg {secs}] time {time filter} 967 {type} {parent regex} {child regex} {attribute regex} 968 [profile {profile name}] [any|all|not group {group name} ...] 969 970 Args: 971 period (str): time period to retrieve series from (default: 'last1h') 972 time_interval (int): interval in seconds for series data points (default: 60) 973 device (str): device name or pattern to match (default: '*') 974 attribute (str): attribute name or pattern to match (default: '*') 975 get_dict (bool): return each row as a dictionary keyed by the 976 header row, rather than the CSV as sent (default: True). 977 The header carries one column heading per interval, which is 978 the time axis for the values under it. As dictionaries those 979 headings are the keys; as lists the header is the first entry, 980 so the list form has one row more than the dictionary form 981 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 982 groups (list): list of group names to filter by (if any) 983 Returns: 984 A list of series data rows (as dictionaries or lists), or None if no data found 985 Raises: 986 AkipsError: if the AKiPS server returns an error 987 """ 988 params = { 989 "cmds": f"cseries interval avg {time_interval} time {period} * {device} * {attribute}" 990 } 991 if groups: 992 group_list = " ".join(groups) 993 params["cmds"] += f" {group_filter} group {group_list}" 994 text = self._get(params=params) 995 if text: 996 # Rows as dictionaries keyed by the header row, or as the CSV as 997 # sent with that header kept as the first entry. The header is 998 # the time axis, one column heading per interval, so the list form 999 # keeps it: without it the values underneath are readings with no 1000 # timestamps. The dictionary form does not need it separately 1001 # because those headings became its keys. 1002 csv_to_list = self._parse_csv(text, header=get_dict) 1003 data_rows = csv_to_list if get_dict else csv_to_list[1:] 1004 if not data_rows: 1005 # A header with nothing under it is an axis with no series on 1006 # it, which is nothing found rather than a result 1007 return None 1008 logger.debug("Found {} series entries".format(len(csv_to_list))) 1009 return csv_to_list 1010 return None 1011 1012 def get_latest_values( 1013 self, 1014 attribute: str, 1015 device: str = "*", 1016 child: str = "*", 1017 period: str = "last1h", 1018 time_interval: int = 300, 1019 group_filter: str = "any", 1020 groups: list[str] | None = None, 1021 ) -> dict[str, dict[str, dict[str, Any]]] | None: 1022 """ 1023 Pull the most recent reading of a numeric attribute for each device. 1024 1025 Numeric attributes do not hold a reading in the config database that 1026 get_attributes() reads; that holds the counter or gauge definition, 1027 which is the same for every device. The readings live in the time 1028 series database, so this asks for a short series and keeps the last 1029 value in it. 1030 1031 The final interval of a series is usually still being filled and comes 1032 back empty, so the last column is not the answer; this returns the 1033 last column that has a value, along with when it was measured. Values 1034 are already scaled by AKiPS, so what comes back is in the attribute's 1035 real units. 1036 1037 Supporting AKiPS command syntax: 1038 1039 cseries [interval total|avg {secs}] time {time filter} 1040 {type} {parent regex} {child regex} {attribute regex} 1041 [profile {profile name}] [any|all|not group {group name} ...] 1042 1043 Args: 1044 attribute (str): the attribute to read 1045 device (str): device name or pattern to match (default: '*') 1046 child (str): child name or pattern to match (default: '*') 1047 period (str): how far back to look (default: 'last1h'). It only 1048 has to be long enough to contain one completed interval 1049 time_interval (int): seconds per interval (default: 300) 1050 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 1051 groups (list): list of group names to filter by (if any) 1052 Returns: 1053 A dictionary of device names to child names to the reading, each 1054 with 'value', 'time' and 'attribute'. A device with no reading in 1055 the period is present with a value of None rather than dropped. 1056 None if nothing matched at all 1057 Raises: 1058 AkipsError: if the AKiPS server returns an error 1059 """ 1060 params = { 1061 "cmds": f"cseries interval avg {time_interval} time {period} " 1062 f"* {device} {child} {attribute}" 1063 } 1064 if groups: 1065 # [any|all|not group {group name} ...] 1066 group_list = " ".join(groups) 1067 params["cmds"] += f" {group_filter} group {group_list}" 1068 text = self._get(params=params) 1069 if not text: 1070 return None 1071 1072 # The columns every cseries reply starts with, before the timestamps 1073 fixed_columns = ("parent", "child", "child description", "attribute") 1074 data: dict[str, dict[str, dict[str, Any]]] = {} 1075 unreadable = [] 1076 rows = cast(list[dict[str, str]], self._parse_csv(text, header=True)) 1077 for row in rows: 1078 parent = row.get("parent") 1079 child_name = row.get("child") 1080 if not parent or not child_name: 1081 continue 1082 # Everything after the fixed columns is a timestamped reading, in 1083 # order, because the reader keeps the header's column order 1084 readings = [ 1085 (column, value) 1086 for column, value in row.items() 1087 if column not in fixed_columns and value 1088 ] 1089 entry: dict[str, Any] = { 1090 "attribute": row.get("attribute", attribute), 1091 "value": None, 1092 "time": None, 1093 } 1094 if readings: 1095 column, value = readings[-1] 1096 try: 1097 entry["value"] = float(value) 1098 except ValueError: 1099 unreadable.append(f"{parent} {child_name} = {value}") 1100 continue 1101 try: 1102 entry["time"] = pytz.timezone(self.server_timezone).localize( 1103 datetime.strptime(column, "%Y-%m-%d %H:%M") 1104 ) 1105 except ValueError: 1106 # A column heading in a shape this does not recognize is 1107 # not worth losing the reading over 1108 entry["time"] = None 1109 data.setdefault(parent, {})[child_name] = entry 1110 1111 if unreadable: 1112 logger.warning( 1113 "Could not read {} of {} {} values from akips, those are " 1114 "missing from the result. First: {}".format( 1115 len(unreadable), 1116 len(unreadable) + len(rows), 1117 attribute, 1118 unreadable[0], 1119 ) 1120 ) 1121 if not data: 1122 return None 1123 logger.debug("Found readings for {} devices".format(len(data))) 1124 return data 1125 1126 def get_aggregate( 1127 self, 1128 period: str = "last1h", 1129 device: str = "*", 1130 attribute: str = "*", 1131 operator: str = "avg", 1132 time_interval: int = 300, 1133 group_filter: str = "any", 1134 groups: list[str] | None = None, 1135 labeled: bool = False, 1136 ) -> list[str] | list[dict[str, Any]] | None: 1137 """ 1138 Pull aggregate counter values over a period of time with optional filtering 1139 by device, attribute, and/or group membership. Defaults to all devices 1140 and attributes over the last hour with average aggregation every 300 seconds. Review 1141 AKiPS documentation for details on time filter syntax. 1142 1143 The aggregate collapses every matching device into a single series, so 1144 unlike get_series() the reply says nothing about what was measured. 1145 AKiPS sends no timestamps with it either, only the numbers, and there 1146 is one more of them than there are intervals: an hour at 300 seconds 1147 returns 13 values, not 12, the last of them landing on the end of the 1148 window. 1149 1150 Pass labeled=True to get a time against each value. That asks the 1151 server for the window with 'tf pairs' and spaces the values across it, 1152 which costs one extra request and is the only honest way to do it, 1153 since computing the axis here would be this module's clock rather than 1154 the server's. 1155 1156 An interval the server has no reading for comes back empty and is kept 1157 with a value of None, so the points stay in step with the axis. A 1158 calendar relative period returns a great many of those: 'last1d' is the 1159 whole of today, so at 300 seconds it is 288 intervals of which only the 1160 elapsed ones hold anything, and the rest are timestamped into the 1161 evening to come. Use 'last24h' for a rolling day with data throughout. 1162 1163 Supporting AKiPS command syntax: 1164 1165 aggregate [interval total|avg {secs}] time {time filter} 1166 {type} {parent regex} {child regex} {attribute regex} 1167 [profile {profile name}] [any|all|not group {group name} ...] 1168 1169 Args: 1170 period (str): time period to retrieve series from (default: 'last1h') 1171 device (str): device name or pattern to match (default: '*') 1172 attribute (str): attribute name or pattern to match (default: '*') 1173 operator (str): aggregation operator, 'avg' or 'total seconds' (default: 'avg') 1174 time_interval (int): seconds per aggregation point (default: 300). 1175 Named to match get_series() and get_latest_values(), which 1176 take the same thing 1177 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 1178 groups (list): list of group names to filter by (if any) 1179 labeled (bool): put a time against each value (default: False). 1180 Adds a request, and needs a period covering one continuous 1181 range; a filter such as 'lastweek; mon to fri 8:00 to 17:00' 1182 is several disjoint ranges and cannot be one axis 1183 Returns: 1184 A list of aggregate values, one more than the number of intervals, 1185 or None if no data found. With labeled=True, a list of 1186 dictionaries with 'time' and 'value', where 'time' is timezone 1187 aware in the server's timezone and 'value' is a float, or None 1188 where the value was not a number 1189 Raises: 1190 ValueError: if labeled is asked for and the period does not 1191 describe one continuous range 1192 AkipsError: if the AKiPS server returns an error 1193 """ 1194 params = { 1195 "cmds": f"aggregate interval {operator} {time_interval} time {period} * {device} * {attribute}" 1196 } 1197 if groups: 1198 group_list = " ".join(groups) 1199 params["cmds"] += f" {group_filter} group {group_list}" 1200 text = self._get(params=params) 1201 if text: 1202 # One CSV row of values, followed by a blank line 1203 rows = cast(list[list[str]], self._parse_csv(text)) 1204 values = rows[0] if rows else [] 1205 if not values: 1206 return None 1207 logger.debug("Found {} aggregate values".format(len(values))) 1208 if not labeled: 1209 return values 1210 return self._label_aggregate(values, period, time_interval) 1211 return None 1212 1213 def _label_aggregate( 1214 self, values: list[str], period: str, time_interval: int 1215 ) -> list[dict[str, Any]]: 1216 """ 1217 Put a time against each value of an aggregate. 1218 1219 The bounds come from the server rather than from arithmetic here, so 1220 the axis is the window AKiPS actually measured. 'tf pairs' answers 1221 with '{start},{end}' in epoch seconds, one line per continuous range 1222 within the filter. 1223 """ 1224 text = self._get(params={"cmds": f"tf pairs {period}"}) 1225 ranges = self._parse_lines(text or "") 1226 if len(ranges) != 1: 1227 raise ValueError( 1228 "Cannot label aggregate values for period {!r}: it describes " 1229 "{} separate ranges, and a single time axis needs one".format( 1230 period, len(ranges) 1231 ) 1232 ) 1233 try: 1234 start_epoch, _end_epoch = (int(field) for field in ranges[0].split(",")) 1235 except ValueError: 1236 raise ValueError( 1237 "Could not read the window for period {!r} from {!r}; expected " 1238 "'{{start}},{{end}}' in epoch seconds".format(period, ranges[0]) 1239 ) from None 1240 1241 timezone = pytz.timezone(self.server_timezone) 1242 labeled = [] 1243 unreadable = [] 1244 empty = 0 1245 for index, value in enumerate(values): 1246 # Kept rather than dropped either way: leaving a point out would 1247 # shift every one after it along the axis 1248 reading: float | None = None 1249 if not value.strip(): 1250 # An interval the server has no data for. A calendar relative 1251 # period such as 'last1d' covers the whole day, so every bucket 1252 # after the current moment comes back empty; that is the period 1253 # doing what it says rather than anything wrong with the reply 1254 empty += 1 1255 else: 1256 try: 1257 reading = float(value) 1258 except ValueError: 1259 unreadable.append(value) 1260 labeled.append( 1261 { 1262 "time": datetime.fromtimestamp( 1263 start_epoch + index * time_interval, tz=timezone 1264 ), 1265 "value": reading, 1266 } 1267 ) 1268 if empty: 1269 logger.debug( 1270 "{} of {} aggregate intervals had no data, kept with a value " 1271 "of None".format(empty, len(values)) 1272 ) 1273 if unreadable: 1274 logger.warning( 1275 "Could not read {} of {} aggregate values as numbers, those " 1276 "points are kept with a value of None. First: {!r}".format( 1277 len(unreadable), len(values), unreadable[0][:100] 1278 ) 1279 ) 1280 return labeled 1281 1282 # Low-level operations, kept for compatibility 1283 1284 def cmd(self, cmd: str, output: str = "raw") -> str | None: 1285 """ 1286 Deprecated since 1.0.0, use call() instead, which reaches every API 1287 section and can parse the reply rather than only returning it raw. 1288 1289 Args: 1290 cmd (str): AKiPS command string to send 1291 output (str): desired output format, only 'raw' is supported 1292 Returns: 1293 The command output, or None if no output 1294 Raises: 1295 ValueError: if an invalid output format is provided 1296 AkipsError: if the AKiPS server returns an error 1297 """ 1298 warnings.warn( 1299 "cmd() is deprecated and will be removed in a future release, " 1300 "use call() instead", 1301 DeprecationWarning, 1302 stacklevel=2, 1303 ) 1304 if output != "raw": 1305 raise ValueError("Invalid output value provided to cmd.") 1306 return cast("str | None", self.call(command=cmd)) 1307 1308 # --------------------------------------------------------------------------- 1309 # api-script methods, these require the 'api-rw' user 1310 1311 def get_device_by_ip(self, ipaddr: str) -> str | None: 1312 """ 1313 Return the device name (primary key) for a device matching the given IP address. 1314 AKiPS records additional IP addresses when found on devices, so this function 1315 can be used to find the primary device name (primary key) from any known IP address. 1316 1317 A device is stored under one address, but it may answer on several. 1318 The case this exists for is a syslog message or SNMP trap arriving 1319 from an interface other than the one AKiPS knows the device by, where 1320 the source address matches no device at all. AKiPS keeps an internal 1321 address to device table, which the site script reads, so this maps any 1322 address the server has seen back to the device holding it. 1323 1324 That table is not the device's attributes, and searching the attributes 1325 is not a substitute. AKiPS keeps the addresses configured on a device 1326 in a CSV file rather than in its database, which is why an mget against 1327 attributes does not find them; the GUI shows the same file as its 1328 'Device to IP Mapping' report. Confirmed on a live server, where an 1329 address resolved here to a device whose entire attribute tree contained 1330 no mention of it. 1331 1332 This is the one read in this module that a read only deployment cannot 1333 perform. AKiPS exposes it as a site script rather than a database 1334 query, so it lives in api-script and needs rw_password even though it 1335 changes nothing. A client holding only ro_password raises 1336 AkipsCredentialError rather than returning None, so a caller building 1337 on it should not plan for a read only deployment. 1338 1339 **This call is sent as GET, not POST**, so its password travels in 1340 the query string. api-script does not answer a POST: the server sends 1341 no body and holds the connection open until the client gives up. See 1342 SECTION_METHODS, which is where to say so if a server of yours does 1343 take a POST on this section. 1344 1345 Supporting AKiPS site script function (which requires the api-rw user): 1346 1347 web_find_device_by_ip(ipaddr) 1348 1349 Args: 1350 ipaddr (str): IP address to search for 1351 Returns: 1352 the device name (str) if found, or None if no match is found 1353 Raises: 1354 AkipsError: if the AKiPS server returns an error 1355 """ 1356 params = {"function": "web_find_device_by_ip", "ipaddr": ipaddr} 1357 text = self._get(section="api-script", params=params) 1358 if not text: 1359 return None 1360 for line in text.split("\n"): 1361 match = re.match(r"IP Address (\S+) is configured on (\S+)", line) 1362 if match: 1363 address = match.group(1) 1364 device_name = match.group(2) 1365 logger.debug(f"Found {address} on device {device_name}") 1366 return device_name 1367 # The site script says so in as many words when it finds nothing, so a 1368 # reply that is neither that nor a match did not come from it. The 1369 # likeliest cause is the script not being installed, which would 1370 # otherwise read as 'no device has that address' and be believed. 1371 if "is not configured on any devices" not in text: 1372 logger.warning( 1373 "web_find_device_by_ip returned something unexpected; check " 1374 "that the site script is installed on this AKiPS server. " 1375 "Reply: {}".format(self._redact_text(text.strip()[:200])) 1376 ) 1377 return None 1378 1379 def set_group_membership(self, device: str, group: str, mode: str) -> None: 1380 """ 1381 Update manual grouping rules for a device, including the special 'maintenance_mode' 1382 group. The web api script fails silently if the device or group does not exist. 1383 1384 **This call is sent as GET, not POST**, so its password travels in 1385 the query string. api-script does not answer a POST: the server sends 1386 no body and holds the connection open until the client gives up. See 1387 SECTION_METHODS, which is where to say so if a server of yours does 1388 take a POST on this section. 1389 1390 Supporting AKiPS site script function (which requires the api-rw user): 1391 1392 web_manual_grouping(type, group, mode, device) 1393 1394 Args: 1395 device (str): the AKiPS name of one device, exactly; this 1396 takes no pattern 1397 group (str): group name to update 1398 mode (str): 'assign' to add device to group, 'clear' to remove device from group 1399 Returns: 1400 None 1401 Raises: 1402 ValueError: if invalid parameters are provided 1403 AkipsError: if the AKiPS server returns an error 1404 """ 1405 if not device: 1406 raise ValueError( 1407 "a valid device name must be provided for manual grouping update" 1408 ) 1409 if not group: 1410 raise ValueError( 1411 "a valid group name must be provided for manual grouping update" 1412 ) 1413 if mode not in ("assign", "clear"): 1414 raise ValueError( 1415 "mode must be 'assign' or 'clear' for manual grouping update" 1416 ) 1417 params = { 1418 "function": "web_manual_grouping", 1419 "type": "device", 1420 "group": group, # group_name 1421 "mode": mode, # 'assign' or 'clear' for device memberships 1422 "device": device, # device_name 1423 } 1424 text = self._get(section="api-script", params=params) 1425 if text: 1426 logger.error("Web API request failed: {}".format(text)) 1427 raise AkipsError(message=text) 1428 return None 1429 1430 SCRIPT_TIMEOUT = 300 1431 """Seconds a site script that does work is given, in place of the 1432 client's timeout. 1433 1434 Site scripts divide into two kinds. Most answer a question and return at 1435 once — get_device_by_ip() and set_group_membership() are ordinary 1436 requests and keep the client's timeout, so a hung one fails as promptly 1437 as any other call. A few go away and do something: deleting a device 1438 today, and discovery, rewalk and rename if those are ever wrapped. Those 1439 are what this is for. 1440 1441 It is not per method on purpose. Every long running script wants the 1442 same thing — more room than a read gets — and a constant for each would 1443 be a new name to learn for every script added. A method needing 1444 something different takes a timeout argument instead. 1445 1446 Why generous: a timeout part way through work that changes the server 1447 leaves the worst of the three outcomes, where the caller cannot tell 1448 whether it happened, since nothing can confirm an outcome when the call 1449 itself raises. Waiting longer costs only waiting. 1450 1451 How long any of these really take is not known. The one timing on 1452 record, a delete just past 30 seconds against a 30 second timeout, was 1453 taken while api-script still hung on every POST, so it measures the 1454 client giving up rather than the work — see SECTION_METHODS. 1455 1456 A client configured with a longer timeout than this keeps it; this is a 1457 floor, not a ceiling.""" 1458 1459 def delete_device(self, device: str, timeout: int | None = None) -> bool: 1460 """ 1461 Delete one device from AKiPS. 1462 1463 **This cannot be undone.** Whether the samples, events and 1464 availability held against the device go with it is a property of 1465 AKiPS's own config_delete_device built in, which the site script calls 1466 and this module cannot see into, so treat the whole record as lost 1467 until AKiPS says otherwise. There is no merge: where the same box is 1468 registered twice under two names, copy whatever the surviving record 1469 should keep before deleting the other one, because nothing moves 1470 across on its own. 1471 1472 **This call is sent as GET, not POST**, so its password travels in 1473 the query string. api-script does not answer a POST: the server sends 1474 no body and holds the connection open until the client gives up. See 1475 SECTION_METHODS, which is where to say so if a server of yours does 1476 take a POST on this section. 1477 1478 Supporting AKiPS site script function (which requires the api-rw user): 1479 1480 web_delete_device(device_names) 1481 1482 AKiPS publishes that script and does not install it by default; see 1483 akips_setup/README.md. It prints nothing whether it worked or not, so 1484 this method confirms the outcome rather than trusting the silence. It 1485 checks the device is there first, which is how a name that never 1486 existed is told apart from one that was removed, and checks it is gone 1487 afterwards, which is how a script that quietly did nothing is caught. 1488 That costs two extra requests, which is the right trade for an 1489 operation with no undo. 1490 1491 **An exception does not mean nothing happened.** The confirmation 1492 below cannot run when the call itself fails, and AKiPS finishes the 1493 work whether or not the client is still listening: a delete that ran 1494 just past a 30 second timeout removed the device and raised anyway, 1495 so the caller recorded a failure against a device already gone. On any exception, ask AKiPS again rather than 1496 recording a failure. Gone, still there, and could not tell are three 1497 different outcomes and only the first two are knowable from here. 1498 1499 Args: 1500 device (str): the AKiPS name of one device, exactly. This takes 1501 no pattern and no list. A name holding a comma or an asterisk 1502 is refused: the site script splits its argument on commas, so 1503 such a name would delete more than was asked for, and a 1504 partial or oversized delete cannot be walked back. 1505 timeout (int): seconds to wait for the delete itself. Defaults 1506 to SCRIPT_TIMEOUT, or the client's timeout if that is longer, 1507 because a timeout during a destructive call leaves an outcome 1508 nobody can read. The two lookups either side are ordinary 1509 reads and use the client's timeout. 1510 Returns: 1511 True if the device was deleted, False if there was no such device. 1512 The two are distinguishable on purpose, so a caller does not 1513 report success for a name that was never there. 1514 Raises: 1515 ValueError: if device is empty, is a pattern, or could name more 1516 than one device 1517 AkipsCredentialError: if no rw_password was given to AKIPS() 1518 AkipsError: if AKiPS returns an error, or if the device is still 1519 present afterwards 1520 """ 1521 if not device: 1522 raise ValueError("a device name must be provided to delete a device") 1523 if device.startswith("/") and device.endswith("/") and len(device) > 1: 1524 raise ValueError( 1525 "delete_device takes one device name, not a pattern. Got " 1526 "{!r}".format(device) 1527 ) 1528 for char in (",", "*"): 1529 if char in device: 1530 # web_delete_device does cgi_param("device_names") in scalar 1531 # context and splits on commas itself, so a comma here is not 1532 # an odd name but a second device. Refused rather than 1533 # escaped, because there is no undo to fall back on. 1534 raise ValueError( 1535 "refusing to delete {!r}: a name containing {!r} can match " 1536 "more than one device, and this cannot be undone".format( 1537 device, char 1538 ) 1539 ) 1540 1541 # Checked before anything is looked up, so a client with no rw 1542 # password fails on the credential rather than after spending a 1543 # request on a delete it could never have made. 1544 self._credentials_for("api-script") 1545 1546 if self.get_device(device) is None: 1547 logger.info("No AKiPS device named {!r}, nothing to delete".format(device)) 1548 return False 1549 1550 params = { 1551 "function": "web_delete_device", 1552 "device_names": device, # one name; the script splits on commas 1553 } 1554 if timeout is None: 1555 # A floor rather than a replacement: a client deliberately given 1556 # longer than this keeps it. 1557 timeout = max(self.timeout, self.SCRIPT_TIMEOUT) 1558 text = self._get(section="api-script", params=params, timeout=timeout) 1559 if text: 1560 logger.error("Web API request failed: {}".format(text)) 1561 raise AkipsError(message=text) 1562 1563 if self.get_device(device) is not None: 1564 raise AkipsError( 1565 message=( 1566 "AKiPS still holds a device named {!r} after " 1567 "web_delete_device returned nothing. Check that the site " 1568 "script is installed and that api-rw is allowed to run " 1569 "it".format(device) 1570 ) 1571 ) 1572 logger.info("Deleted AKiPS device {!r} and its history".format(device)) 1573 return True 1574 1575 # --------------------------------------------------------------------------- 1576 # api-msg methods, these require the 'api-ro' user 1577 1578 # The message types AKiPS keeps, and what get_msg accepts. None asks for 1579 # both, which is what the api-msg section returns when the parameter is 1580 # left off. 1581 MSG_TYPES = ("syslog", "trap") 1582 """The message types get_msg() accepts. None asks for both.""" 1583 1584 # 'period' and 'msg_type' map to the AKiPS query parameters 'time' and 1585 # 'type'. They are deliberately named apart from those, because 'type' is 1586 # a builtin and 'time' a standard library module, and because 'period' is 1587 # what the rest of this module already calls a time filter. 1588 def get_msg( 1589 self, 1590 period: str = "last1h", 1591 addr: str | None = None, 1592 msg_type: str | None = None, 1593 device: str | None = None, 1594 regex: str | None = None, 1595 limit: int | None = None, 1596 ) -> list[dict[str, str]] | None: 1597 """ 1598 Retrieve syslog or trap messages from the AKiPS api-msg database. The api-msg 1599 access requires username to be 'api-ro'. 1600 1601 Supporting AKiPS web API syntax: 1602 1603 https://{server}/api-msg?password={pw};time={time filter}; 1604 [addr={ip filter}];[type=syslog|trap];[device={name}|{regex}]; 1605 [regex={regex filter}];[limit={qty messages}] 1606 1607 This is the highest volume call here, and worth filtering. Measured on 1608 a 17,000 device fleet, an unfiltered 'last1h' returned 472,014 messages 1609 in 5.5 seconds; the same hour asking only for traps returned 5,160 in 1610 0.8 seconds. Syslog is the bulk of it, and a single appliance can be a 1611 large share of that on its own. See get_traps() and get_syslog(). 1612 1613 Args: 1614 period (str): Required, time period to retrieve messages from 1615 (default: 'last1h'). 'lastNm' and 'lastNh' are rolling 1616 windows of the length they name, measured back from the moment 1617 of the call. 'lastNd' is calendar relative, meaning N-1 whole 1618 days plus today so far, so 'last1d' is today rather than 24 1619 hours; use 'last24h' for a rolling day 1620 addr (str): IP address to filter messages by (default: None) 1621 msg_type (str): message type, 'syslog' or 'trap', or None for both 1622 (default: None). See get_syslog() and get_traps(), which name 1623 the type rather than asking a caller to spell it 1624 device (str): device name to filter messages by (default: None) 1625 regex (str): regex pattern to filter message content by (default: None) 1626 limit (int): maximum number of messages to return (default: None). 1627 AKiPS fills this from the start of the window, so by default it 1628 returns the oldest matching messages rather than the newest, 1629 and there is no ordering parameter on the request. AKiPS 25.6 1630 added a reverse sort option under Miscellaneous Settings, which 1631 is server wide rather than per call; whether it reaches this 1632 section has not been tested here. For recent activity narrow 1633 'period' instead: 'last15m' with no limit costs far less than 1634 an hour of messages thrown away after the fact 1635 Returns: 1636 A list of dictionaries, each with 'time', 'type', 'ip_ver', 1637 'ip_addr' and 'message', or None if no data found. 'ip_addr' is 1638 where the message came from, which need not be the address AKiPS 1639 holds for the device: a device with several interfaces can send 1640 from any of them. get_device_by_ip() resolves one to a device 1641 Raises: 1642 ValueError: if msg_type is not 'syslog', 'trap' or None 1643 AkipsError: if the AKiPS server returns an error 1644 """ 1645 1646 # Checked rather than quietly ignored. An unrecognized type used to be 1647 # dropped, so a caller asking for 'traps' or 'Syslog' was sent no type 1648 # at all and got both back believing it had filtered to one. 1649 if msg_type is not None and msg_type not in self.MSG_TYPES: 1650 raise ValueError( 1651 "Invalid msg_type provided to get_msg, expected one of {}, " 1652 "or None for both".format(", ".join(self.MSG_TYPES)) 1653 ) 1654 params = {"time": period} 1655 if msg_type is not None: 1656 params["type"] = msg_type 1657 if addr: 1658 params["addr"] = addr 1659 if device: 1660 params["device"] = device 1661 if regex: 1662 params["regex"] = regex 1663 if limit: 1664 params["limit"] = str(limit) 1665 text = self._get(section="api-msg", params=params) 1666 if text: 1667 # Each syslog or trap message contains: 1668 # header line: {system timestamp} {type} {IP version} {IP address} 1669 # message line(s): {message text} 1670 # blank terminating line 1671 # 1672 # Records are split on that blank line rather than by recognizing 1673 # each header, because a body line can look exactly like a header 1674 # and would otherwise start a new record in the middle of a 1675 # message, turning one message into two with empty bodies. 1676 data = [] 1677 unparsed = 0 1678 for record in re.split(r"\n\s*\n", text): 1679 lines = [line for line in record.split("\n") if line.strip()] 1680 if not lines: 1681 continue 1682 header = re.match( 1683 r"^(?P<time>\S+)\s(?P<type>\S+)\s(?P<ip_ver>[46])\s(?P<ip_addr>\S+)$", 1684 lines[0], 1685 ) 1686 if not header: 1687 unparsed += 1 1688 continue 1689 data.append( 1690 { 1691 "time": header.group("time"), 1692 "type": header.group("type"), 1693 "ip_ver": header.group("ip_ver"), 1694 "ip_addr": header.group("ip_addr"), 1695 # Everything after the header is the message, whatever 1696 # any of those lines happen to look like 1697 "message": "\n".join(lines[1:]), 1698 } 1699 ) 1700 if unparsed: 1701 logger.warning( 1702 "Could not parse {} of {} message records from akips, " 1703 "those messages are missing from the result".format( 1704 unparsed, unparsed + len(data) 1705 ) 1706 ) 1707 # A reply that parses to nothing is nothing found, the same 1708 # answer an empty reply gives, rather than an empty container 1709 if not data: 1710 return None 1711 logger.debug("Found {} messages in akips".format(len(data))) 1712 return data 1713 return None 1714 1715 def get_syslog( 1716 self, 1717 period: str = "last1h", 1718 addr: str | None = None, 1719 device: str | None = None, 1720 regex: str | None = None, 1721 limit: int | None = None, 1722 ) -> list[dict[str, str]] | None: 1723 """ 1724 Retrieve syslog messages, leaving traps out. 1725 1726 The same as get_msg(msg_type='syslog') with every other filter 1727 forwarded, named so the type does not have to be spelled correctly to 1728 take effect. 1729 1730 Syslog is the high volume half of api-msg: an unfiltered hour was 1731 465,936 messages on a 17,000 device fleet, one appliance accounting 1732 for a large share of it. Pass a shorter period, a device or a regex 1733 unless the whole of it is wanted. 1734 1735 Args: 1736 period (str): time period to retrieve messages from 1737 (default: 'last1h'). 'lastNm' and 'lastNh' are rolling 1738 windows of the length they name; 'lastNd' is calendar 1739 relative, so 'last1d' is today rather than 24 hours 1740 addr (str): IP address to filter messages by (default: None) 1741 device (str): device name to filter messages by (default: None) 1742 regex (str): regex pattern to filter message content by 1743 (default: None) 1744 limit (int): maximum number of messages to return (default: None). 1745 This returns the oldest matching messages by default, not the 1746 newest; narrow 'period' for recent activity. See get_msg() 1747 for the server setting that may reverse it 1748 Returns: 1749 A list of dictionaries, each with 'time', 'type', 'ip_ver', 1750 'ip_addr' and 'message', or None if no data found. 'ip_addr' is 1751 where the message came from, which need not be the address AKiPS 1752 holds for the device: a device with several interfaces can send 1753 from any of them. get_device_by_ip() resolves one to a device 1754 Raises: 1755 AkipsError: if the AKiPS server returns an error 1756 """ 1757 return self.get_msg( 1758 period=period, 1759 msg_type="syslog", 1760 addr=addr, 1761 device=device, 1762 regex=regex, 1763 limit=limit, 1764 ) 1765 1766 def get_traps( 1767 self, 1768 period: str = "last1h", 1769 addr: str | None = None, 1770 device: str | None = None, 1771 regex: str | None = None, 1772 limit: int | None = None, 1773 ) -> list[dict[str, str]] | None: 1774 """ 1775 Retrieve SNMP traps, leaving syslog out. 1776 1777 The same as get_msg(msg_type='trap') with every other filter 1778 forwarded, named so the type does not have to be spelled correctly to 1779 take effect. 1780 1781 Asking for traps is what makes this call cheap enough to poll: on a 1782 17,000 device fleet an hour of traps was 5,160 messages against 1783 472,014 for an unfiltered hour. 1784 1785 The body of a trap is a varbind list, one per line, as 1786 '{module} {attribute} {instance} {type} {value}'. It is returned as 1787 the raw 'message' text; this does not split it up. 1788 1789 Args: 1790 period (str): time period to retrieve messages from 1791 (default: 'last1h'). 'lastNm' and 'lastNh' are rolling 1792 windows of the length they name; 'lastNd' is calendar 1793 relative, so 'last1d' is today rather than 24 hours 1794 addr (str): IP address to filter messages by (default: None) 1795 device (str): device name to filter messages by (default: None) 1796 regex (str): regex pattern to filter message content by 1797 (default: None) 1798 limit (int): maximum number of messages to return (default: None). 1799 This returns the oldest matching messages by default, not the 1800 newest; narrow 'period' for recent activity. See get_msg() 1801 for the server setting that may reverse it 1802 Returns: 1803 A list of dictionaries, each with 'time', 'type', 'ip_ver', 1804 'ip_addr' and 'message', or None if no data found. 'ip_addr' is 1805 where the message came from, which need not be the address AKiPS 1806 holds for the device: a device with several interfaces can send 1807 from any of them. get_device_by_ip() resolves one to a device 1808 Raises: 1809 AkipsError: if the AKiPS server returns an error 1810 """ 1811 return self.get_msg( 1812 period=period, 1813 msg_type="trap", 1814 addr=addr, 1815 device=device, 1816 regex=regex, 1817 limit=limit, 1818 ) 1819 1820 # --------------------------------------------------------------------------- 1821 # api-availability methods, these require the 'api-ro' user 1822 1823 # AKiPS has two kinds of time filter and they are easy to confuse. 1824 # 'lastNd' is calendar relative: it means N-1 whole days plus today so 1825 # far, so 'last1d' is today and measures minutes just after midnight. 1826 # 'lastNh' and 'lastNm' are rolling windows of the length they name. 1827 # 1828 # These methods default to the rolling form. An availability figure is a 1829 # percentage of the window it was measured over, and a caller asking for 1830 # 'the last day' and rendering the answer should not silently get a 1831 # five minute sample that reads as a reliable 100% every night. 1832 AVAILABILITY_PERIOD = "last24h" 1833 """The period the availability methods use by default, a rolling 24 1834 hours rather than 'last1d', which AKiPS reads as today so far.""" 1835 1836 def get_group_availability( 1837 self, 1838 period: str = AVAILABILITY_PERIOD, 1839 report: str = "ping4", 1840 group: str | None = None, 1841 ) -> list[dict[str, str]] | None: 1842 """ 1843 Retrieve availability statistics for a group of devices over a time period. 1844 1845 # output format: {child},{attr},{group name},{total time},{match time},{group target},{tf}[;{group tf}] 1846 # example: nm-availability mode group time last1w report ping4 1847 1848 ping4,PING.icmpState,1-Building-4,11688115,11687711,9990,last1w 1849 ping4,PING.icmpState,1-Fraser,8213270,8213190,9990,last1w 1850 ping4,PING.icmpState,1-Building-16,44541195,44540002,9990,last1w 1851 ping4,PING.icmpState,Accedian,1766635,1766635,9890,last1w;mon to sat 6:00 to 20:00 1852 ping4,PING.icmpState,Aerohive,589475,589475,9999,last1w;mon to fri 7:00 to 19:00; sat 8:00 to 18:00 1853 1854 Args: 1855 period (str): time filter, refer to the AKiPS programming guide 1856 (default: 'last24h'). 'lastNd' is calendar relative, meaning 1857 N-1 whole days plus today so far, so 'last1d' is today rather 1858 than 24 hours and shrinks to minutes just after midnight. 1859 'lastNh' and 'lastNm' are rolling windows of the length they 1860 name. 'total time' in the reply is the window measured 1861 report (str): 'ping4', 'ping6', 'snmp' or 'ifstatus', in any 1862 combination, comma separated (default: 'ping4') 1863 group (str): group name to filter by, or every group 1864 Returns: 1865 A list of dictionaries, one per group, or None if nothing matched 1866 Raises: 1867 AkipsError: if the AKiPS server returns an error 1868 """ 1869 params = { 1870 "maintenance": "off", # 'on' or 'off', show/hide maintenance mode devices 1871 "mode": "group", # 'group', 'device' or 'events' 1872 "time": period, # time filter, refer to programming guide 1873 "report": report, # 'ping4', 'ping6', 'snmp', 'ifstatus'. Any combination, comma separated, 1874 # "entity": device, # {device} [{child}] to filter by device or child 1875 "group": group, # {group name} to filter by group 1876 # "profile": "" # {profile name} to filter by profile 1877 } 1878 text = self._get(section="api-availability", params=params) 1879 if text: 1880 # This endpoint sends no header row, so the column names come from 1881 # here rather than from the reply 1882 column_headers = [ 1883 "child", 1884 "attr", 1885 "group name", 1886 "total time", 1887 "match time", 1888 "group target", 1889 "tf", 1890 ] 1891 csv_to_list = self._parse_csv(text, fieldnames=column_headers) 1892 logger.debug("Found {} entries".format(len(csv_to_list))) 1893 return cast("list[dict[str, str]]", csv_to_list) 1894 return None 1895 1896 def get_device_availability( 1897 self, 1898 period: str = AVAILABILITY_PERIOD, 1899 report: str = "ping4", 1900 device: str | None = None, 1901 group: str | None = None, 1902 ) -> list[dict[str, str]] | None: 1903 """ 1904 Retrieve availability statistics per device over a time period. 1905 1906 Where group mode summarises a whole group, this reports each device 1907 and child separately, so a device checked by both ping and SNMP 1908 appears on two rows. 1909 1910 A device or a group is required. Unlike group mode, device mode 1911 answers an unscoped call with an empty body rather than an error, 1912 which would reach the caller as None and read as 'nothing to report'. 1913 1914 # output format: {parent},{child},{attr},{total time},{match time},{group target} 1915 # example: nm-availability mode device time last1w report snmp,ping4 group Accedian 1916 1917 accedian-131-2-7,ping4,PING.icmpState,136020,136020,9890 1918 accedian-131-2-7,sys,SNMP.snmpState,136020,136020,9890 1919 accedian-131-2-8,ping4,PING.icmpState,136020,136020,9890 1920 accedian-131-2-8,sys,SNMP.snmpState,136020,136020,9890 1921 1922 'group target' is the availability AKiPS is configured to expect, in 1923 basis points, so 9890 is 98.90% and 10000 is 100.00%. It is set per 1924 group, so a caller can report against the target already agreed on 1925 the server rather than inventing a threshold of its own. 1926 1927 Args: 1928 period (str): time filter, refer to the AKiPS programming guide 1929 (default: 'last24h'). 'lastNd' is calendar relative, meaning 1930 N-1 whole days plus today so far, so 'last1d' is today rather 1931 than 24 hours and shrinks to minutes just after midnight. 1932 'lastNh' and 'lastNm' are rolling windows of the length they 1933 name. 'total time' in the reply is the window measured 1934 report (str): 'ping4', 'ping6', 'snmp' or 'ifstatus', in any 1935 combination, comma separated (default: 'ping4') 1936 device (str): device to filter by, as '{device}' or 1937 '{device} {child}'. This is the device's AKiPS name 1938 exactly, its one primary key, which is either its sysName or 1939 its IP address depending on how the server names devices, and 1940 takes no pattern; get_device_by_ip() resolves an address to it 1941 group (str): group name to filter by 1942 Returns: 1943 A list of dictionaries, one per device and child, or None if 1944 nothing matched 1945 Raises: 1946 ValueError: if neither device nor group is given 1947 AkipsError: if the AKiPS server returns an error 1948 """ 1949 # Checked before the request, so a call that could only ever come back 1950 # empty fails as the mistake it is rather than as good news 1951 if device is None and group is None: 1952 raise ValueError( 1953 "get_device_availability needs a device or a group to scope it, " 1954 "an unscoped call returns nothing at all" 1955 ) 1956 params = { 1957 "maintenance": "off", # 'on' or 'off', show/hide maintenance mode devices 1958 "mode": "device", # 'group', 'device' or 'events' 1959 "time": period, # time filter, refer to programming guide 1960 "report": report, # 'ping4', 'ping6', 'snmp', 'ifstatus'. Any combination, comma separated, 1961 # 'entity' is not in the nm-availability syntax the AKiPS API 1962 # guide publishes, which lists only mode, time, report, group and 1963 # profile. It works, and is how device and event mode are scoped 1964 # here, but being undocumented it is the parameter most likely to 1965 # change under us in a future AKiPS release. 1966 "entity": device, # {device} [{child}] to filter by device or child 1967 "group": group, # {group name} to filter by group 1968 } 1969 text = self._get(section="api-availability", params=params) 1970 if text: 1971 # This endpoint sends no header row, so the column names come from 1972 # here rather than from the reply. They are not group mode's 1973 # columns; each mode of nm-availability returns its own. 1974 column_headers = [ 1975 "parent", 1976 "child", 1977 "attr", 1978 "total time", 1979 "match time", 1980 "group target", 1981 ] 1982 csv_to_list = self._parse_csv(text, fieldnames=column_headers) 1983 logger.debug("Found {} entries".format(len(csv_to_list))) 1984 return cast("list[dict[str, str]]", csv_to_list) 1985 return None 1986 1987 def get_event_availability( 1988 self, 1989 period: str = AVAILABILITY_PERIOD, 1990 report: str = "ping4", 1991 device: str | None = None, 1992 group: str | None = None, 1993 ) -> list[dict[str, str]] | None: 1994 """ 1995 Retrieve the up and down event pairs behind a device's availability. 1996 1997 Where device mode gives the totals, this gives the outages that 1998 produced them, one row per pair. 1999 2000 # output format: {parent},{child},{down},{up},{total time},{match time} 2001 # example: nm-availability mode events time last1M report ping4 entity cisco-131-16-1 2002 2003 cisco-131-16-1,ping4,1603822871,1603822916,2389764,2388341 2004 cisco-131-16-1,ping4,1603088563,1603089823,2389764,2388341 2005 cisco-131-16-1,ping4,1603060380,1603060498,2389764,2388341 2006 2007 'down' and 'up' are epoch seconds bounding a single outage, so a 2008 device that went down twice comes back as two rows. Both are empty 2009 for a device that stayed up, which still reports the window it was 2010 measured over. 2011 2012 Take the length of an outage as 'up' minus 'down'. 'total time' and 2013 'match time' describe the measurement rather than the row they sit 2014 beside: every row in a reply carries the same 'total time', the 2015 length of the window, and a device's 'match time' is that less the 2016 time it spent down. Measured against a live server, a device with 2017 outages of 44 and 46 seconds came back with a 'match time' 90 below 2018 'total time' on both of its rows, while devices in the same reply 2019 that stayed up had the two equal. 2020 2021 So 'total time' is not the length of the outage on its row. Reading 2022 it that way gives the whole measurement window as the duration of a 2023 one minute flap. 2024 2025 These columns are not the ones group or device mode returns, so the 2026 three modes are parsed separately rather than sharing a field list. 2027 2028 Args: 2029 period (str): time filter, refer to the AKiPS programming guide 2030 (default: 'last24h'). 'lastNd' is calendar relative, meaning 2031 N-1 whole days plus today so far, so 'last1d' is today rather 2032 than 24 hours and shrinks to minutes just after midnight. 2033 'lastNh' and 'lastNm' are rolling windows of the length they 2034 name. 'total time' in the reply is the window measured 2035 report (str): 'ping4', 'ping6', 'snmp' or 'ifstatus', in any 2036 combination, comma separated (default: 'ping4') 2037 device (str): device to filter by, as '{device}' or 2038 '{device} {child}'. This is the device's AKiPS name 2039 exactly, its one primary key, which is either its sysName or 2040 its IP address depending on how the server names devices, and 2041 takes no pattern; get_device_by_ip() resolves an address to it 2042 group (str): group name to filter by 2043 Returns: 2044 A list of dictionaries, one per up and down pair, or None if 2045 nothing matched 2046 Raises: 2047 ValueError: if neither device nor group is given 2048 AkipsError: if the AKiPS server returns an error 2049 """ 2050 # Same as device mode, confirmed against a server: without a scope the 2051 # reply is empty rather than an error, which would arrive as None and 2052 # read as 'no outages' 2053 if device is None and group is None: 2054 raise ValueError( 2055 "get_event_availability needs a device or a group to scope it, " 2056 "an unscoped call returns nothing at all" 2057 ) 2058 params = { 2059 "maintenance": "off", # 'on' or 'off', show/hide maintenance mode devices 2060 "mode": "events", # 'group', 'device' or 'events' 2061 "time": period, # time filter, refer to programming guide 2062 "report": report, # 'ping4', 'ping6', 'snmp', 'ifstatus'. Any combination, comma separated, 2063 # 'entity' is not in the nm-availability syntax the AKiPS API 2064 # guide publishes, which lists only mode, time, report, group and 2065 # profile. It works, and is how device and event mode are scoped 2066 # here, but being undocumented it is the parameter most likely to 2067 # change under us in a future AKiPS release. 2068 "entity": device, # {device} [{child}] to filter by device or child 2069 "group": group, # {group name} to filter by group 2070 } 2071 text = self._get(section="api-availability", params=params) 2072 if text: 2073 # This endpoint sends no header row, so the column names come from 2074 # here rather than from the reply 2075 column_headers = [ 2076 "parent", 2077 "child", 2078 "down", 2079 "up", 2080 "total time", 2081 "match time", 2082 ] 2083 csv_to_list = self._parse_csv(text, fieldnames=column_headers) 2084 logger.debug("Found {} entries".format(len(csv_to_list))) 2085 return cast("list[dict[str, str]]", csv_to_list) 2086 return None 2087 2088 # --------------------------------------------------------------------------- 2089 # Generic operations, these reach any API section 2090 # 2091 # call() is not fixed to one section the way the methods above are. It 2092 # takes the section as an argument and picks the account from 2093 # SECTION_USERS, which is how the sections with no methods of their own 2094 # here are reached. 2095 2096 # The reply shapes call() can parse, mapped to the parser for each 2097 OUTPUT_FORMATS = ("raw", "lines", "key_value", "attributes", "csv", "csv_dict") 2098 """The reply shapes call() can parse.""" 2099 2100 def call( 2101 self, 2102 command: str | None = None, 2103 section: str = "api-db", 2104 params: dict[str, Any] | None = None, 2105 output: str = "raw", 2106 user: str | None = None, 2107 ) -> Any: 2108 """ 2109 Send an arbitrary request to any AKiPS web API section and parse the 2110 reply in one of the shapes AKiPS replies in. 2111 2112 This is the general purpose call for anything the specific methods do 2113 not cover. It parses with the same routines they use, so an ad-hoc 2114 query returns the same shape its dedicated method would. 2115 2116 Sections do not share a parameter vocabulary. api-db takes a command 2117 string, while api-script, api-msg and api-availability each take their 2118 own named parameters, so pass 'command' for the first and 'params' for the 2119 others. Passing both adds the command to the given parameters. 2120 2121 Output formats, and where each one occurs: 2122 2123 raw the reply unchanged, as a string 2124 lines a list of non-blank lines 2125 key_value '{key} = {value}' lines, as from mgroup 2126 attributes '{parent} {child} {attribute} = {value}' lines, as from 2127 mget, nested by parent, child, then attribute 2128 csv CSV rows as lists, for replies with no header row 2129 csv_dict CSV rows as dictionaries keyed by the header row 2130 2131 Args: 2132 command (str): command string for the api-db section, shorthand 2133 for params={'cmds': command} 2134 section (str): API section to call (default: 'api-db') 2135 params (dict): parameters for sections that take no command string 2136 output (str): one of the formats listed above (default: 'raw') 2137 user (str): force the 'ro' or 'rw' account, for a section 2138 whose requirement is not in SECTION_USERS, or a command 2139 needing more rights than its section usually does 2140 Returns: 2141 The reply in the requested shape, or None if the server returned 2142 nothing 2143 Raises: 2144 ValueError: if output is not a supported format, or if neither 2145 command nor params was provided 2146 AkipsError: if the AKiPS server returns an error 2147 """ 2148 # Check before making the request, so a bad argument fails the same way 2149 # whether or not the server returned anything 2150 if output not in self.OUTPUT_FORMATS: 2151 raise ValueError( 2152 "Invalid output value provided to call, expected one of {}".format( 2153 ", ".join(self.OUTPUT_FORMATS) 2154 ) 2155 ) 2156 if command is None and params is None: 2157 raise ValueError("call requires either a command or a params dictionary") 2158 2159 request_params = dict(params or {}) 2160 if command is not None: 2161 request_params["cmds"] = command 2162 2163 text = self._get(section=section, params=request_params, user=user) 2164 if not text: 2165 return None 2166 2167 if output == "raw": 2168 return text 2169 if output == "lines": 2170 return self._parse_lines(text) 2171 if output == "key_value": 2172 return self._parse_key_value(text) 2173 if output == "attributes": 2174 return self._parse_attributes(text) 2175 if output == "csv_dict": 2176 return self._parse_csv(text, header=True) 2177 return self._parse_csv(text) 2178 2179 # --------------------------------------------------------------------------- 2180 # Response parsers 2181 # 2182 # AKiPS replies in a handful of shapes. Each one is parsed in exactly one 2183 # place here, so the specific methods above and the generic call() cannot 2184 # drift apart in how they read the same reply. 2185 2186 @staticmethod 2187 def _parse_lines(text: str) -> list[str]: 2188 """ 2189 Split a reply into its non-blank lines. 2190 2191 Args: 2192 text (str): the raw reply from AKiPS 2193 Returns: 2194 A list of lines with blank ones removed 2195 """ 2196 return [line for line in text.split("\n") if line.strip()] 2197 2198 @staticmethod 2199 def _parse_key_value(text: str) -> dict[str, str]: 2200 """ 2201 Parse lines of '{key} = {value}', the shape mgroup replies in. 2202 2203 Args: 2204 text (str): the raw reply from AKiPS 2205 Returns: 2206 A dictionary of keys to their unsplit values 2207 """ 2208 data = {} 2209 for line in text.split("\n"): 2210 match = re.match(r"^(\S+)\s=\s(.*)$", line) 2211 if match: 2212 data[match.group(1)] = match.group(2) 2213 return data 2214 2215 @staticmethod 2216 def _parse_attributes(text: str) -> dict[str, dict[str, dict[str, str | None]]]: 2217 """ 2218 Parse lines of '{parent} {child} {attribute} = {value}', the shape 2219 mget replies in. An attribute with nothing after the equals has no 2220 value and is recorded as None. 2221 2222 Args: 2223 text (str): the raw reply from AKiPS 2224 Returns: 2225 A nested dictionary of parent, child, attribute to value 2226 """ 2227 data: dict[str, dict[str, dict[str, str | None]]] = {} 2228 unparsed = [] 2229 for line in text.split("\n"): 2230 match = re.match(r"^(\S+)\s(\S+)\s(\S+)\s=(\s(.*))?$", line) 2231 if not match: 2232 # Blank lines are how the reply ends and are not a problem. 2233 # Anything else is the server saying something this cannot 2234 # read, and it must not vanish: every method built on this one 2235 # would otherwise report less than AKiPS sent, with nothing to 2236 # say so. 2237 if line.strip(): 2238 unparsed.append(line) 2239 continue 2240 parent, child, attribute = ( 2241 match.group(1), 2242 match.group(2), 2243 match.group(3), 2244 ) 2245 data.setdefault(parent, {}).setdefault(child, {})[attribute] = match.group( 2246 5 2247 ) 2248 if unparsed: 2249 logger.warning( 2250 "Could not parse {} of {} attribute lines from akips, those " 2251 "values are missing from the result. First: {}".format( 2252 len(unparsed), 2253 len(unparsed) 2254 + sum( 2255 len(attributes) 2256 for children in data.values() 2257 for attributes in children.values() 2258 ), 2259 unparsed[0][:200], 2260 ) 2261 ) 2262 return data 2263 2264 @staticmethod 2265 def _parse_csv( 2266 text: str, fieldnames: list[str] | None = None, header: bool = False 2267 ) -> list[dict[str, str]] | list[list[str]]: 2268 """ 2269 Parse a CSV reply. AKiPS is not consistent about header rows, so the 2270 caller says which shape to expect rather than this guessing. 2271 2272 Args: 2273 text (str): the raw reply from AKiPS 2274 fieldnames (list): column names for a reply that carries no header 2275 header (bool): treat the first row as the header row 2276 Returns: 2277 A list of rows, as dictionaries when column names are known from 2278 either fieldnames or a header row, otherwise as lists 2279 """ 2280 buff = io.StringIO(text) 2281 if fieldnames is not None: 2282 return list(csv.DictReader(buff, fieldnames=fieldnames)) 2283 if header: 2284 return list(csv.DictReader(buff)) 2285 return [row for row in csv.reader(buff) if row] 2286 2287 # --------------------------------------------------------------------------- 2288 # Base operations 2289 2290 def _parse_enum(self, enum_string: str) -> dict[str, Any]: 2291 """ 2292 Attributes with a type of enum return five values separated by commas. 2293 2294 Args: 2295 enum_string (str): the raw enum string from AKiPS 2296 Returns: 2297 A dictionary with keys: number, value, created, modified, description 2298 Raises: 2299 AkipsError: if the provided string is not a valid enum type value 2300 """ 2301 # The trailing description is free text and routinely contains spaces, 2302 # so it takes the rest of the line rather than a non-whitespace run 2303 match = re.match(r"^(\S*),(\S*),(\S*),(\S*),(.*)$", enum_string) 2304 if match: 2305 entry = { 2306 "number": match.group(1), # list number (from MIB) 2307 "value": match.group(2), # text value (from MIB) 2308 # 'created': match.group(3), # time created (epoch timestamp) 2309 # 'modified': match.group(4), # time modified (epoch timestamp) 2310 "description": match.group(5), # child description 2311 } 2312 entry["created"] = datetime.fromtimestamp( 2313 int(match.group(3)), tz=pytz.timezone(self.server_timezone) 2314 ) 2315 entry["modified"] = datetime.fromtimestamp( 2316 int(match.group(4)), tz=pytz.timezone(self.server_timezone) 2317 ) 2318 return entry 2319 else: 2320 raise AkipsError(message=f"Not a ENUM type value: {enum_string}") 2321 2322 def _get_enum_attribute( 2323 self, 2324 attribute: str, 2325 child: str = "*", 2326 values: tuple[str, ...] | list[str] | None = None, 2327 group_filter: str = "any", 2328 groups: list[str] | None = None, 2329 ) -> dict[str, dict[str, Any]] | None: 2330 """ 2331 Pull one enum typed attribute and return it parsed, keyed by device. 2332 2333 Shared by the methods that ask 'which devices are in a bad state', in 2334 which the interesting answer is the enum's text value and when it last 2335 changed. Filtering by value is done by AKiPS rather than here, so a 2336 fleet wide query does not fetch every device to discard most of them. 2337 2338 Supporting AKiPS command syntax: 2339 2340 mget {type} [{parent regex} [{child regex} [{attribute regex}]]] 2341 [descr {/regex/}] [value {text|integer|/regex/}] 2342 [profile {profile name}] [any|all|not group {group name} ...] 2343 2344 Args: 2345 attribute (str): the attribute to read 2346 child (str): child name or pattern to match (default: '*') 2347 values (list): only report these enum values, or None for all 2348 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 2349 groups (list): list of group names to filter by (if any) 2350 Returns: 2351 A dictionary of device names to the parsed enum with the device 2352 'name' and 'child' added, or None if nothing matched 2353 Raises: 2354 AkipsError: if the AKiPS server returns an error 2355 """ 2356 params = {"cmds": f"mget * * {child} {attribute}"} 2357 if values: 2358 # [value {text|/regex/|integer|ipaddr}] 2359 params["cmds"] += " value /{}/".format("|".join(values)) 2360 if groups: 2361 # [any|all|not group {group name} ...] 2362 params["cmds"] += f" {group_filter} group {' '.join(groups)}" 2363 text = self._get(params=params) 2364 if not text: 2365 return None 2366 2367 data: dict[str, dict[str, Any]] = {} 2368 unparsed = [] 2369 for parent, children in self._parse_attributes(text).items(): 2370 for child_name, attributes in children.items(): 2371 for value in attributes.values(): 2372 if value is None: 2373 continue 2374 try: 2375 entry = self._parse_enum(value) 2376 except AkipsError: 2377 # One device reporting something unexpected should not 2378 # cost the answer for every other device 2379 unparsed.append(f"{parent} {child_name} = {value}") 2380 continue 2381 entry["name"] = parent 2382 entry["child"] = child_name 2383 if parent in data: 2384 # Keyed by device, so a device reporting this on more 2385 # than one child would quietly lose all but one 2386 logger.warning( 2387 "{} reports {} on more than one child, " 2388 "keeping {!r} and discarding {!r}".format( 2389 parent, attribute, data[parent]["child"], child_name 2390 ) 2391 ) 2392 continue 2393 data[parent] = entry 2394 if unparsed: 2395 logger.warning( 2396 "Could not parse {} of {} {} values from akips, those devices " 2397 "are missing from the result. First: {}".format( 2398 len(unparsed), len(unparsed) + len(data), attribute, unparsed[0] 2399 ) 2400 ) 2401 if not data: 2402 return None 2403 logger.debug("Found {} devices reporting {}".format(len(data), attribute)) 2404 return data 2405 2406 def _credentials_for( 2407 self, section: str, user: str | None = None 2408 ) -> tuple[str, str]: 2409 """ 2410 Pick the AKiPS account a request should authenticate as. 2411 2412 Args: 2413 section (str): API section being called 2414 user (str): force an account, 'ro' or 'rw', for a section whose 2415 requirement is not known or differs from the usual one 2416 Returns: 2417 A tuple of the username and password to send 2418 Raises: 2419 AkipsCredentialError: if the account this call needs has no password 2420 ValueError: if user names an account that does not exist 2421 """ 2422 if self._account_override is not None: 2423 # A custom account stands in for both 2424 return self._account_override 2425 2426 required = user if user is not None else self.SECTION_USERS.get(section) 2427 if required in ("ro", "api-ro"): 2428 required = "api-ro" 2429 elif required in ("rw", "api-rw"): 2430 required = "api-rw" 2431 elif required is not None: 2432 raise ValueError( 2433 f"Unknown AKiPS account {required!r}, expected 'ro' or 'rw'" 2434 ) 2435 2436 if required is None: 2437 # Either account works here, so use the lesser privileged one 2438 if self.ro_password is not None: 2439 return ("api-ro", self.ro_password) 2440 return ("api-rw", str(self.rw_password)) 2441 2442 password = self.ro_password if required == "api-ro" else self.rw_password 2443 if password is None: 2444 argument = "ro_password" if required == "api-ro" else "rw_password" 2445 raise AkipsCredentialError( 2446 f"{section} requires the {required} account, but no {argument} " 2447 f"was given to AKIPS()" 2448 ) 2449 return (required, password) 2450 2451 # Substrings that mark a name as holding a credential, whether it is a 2452 # query parameter or an AKiPS attribute. Matching loosely is deliberate: 2453 # over redacting costs a value in a debug log, under redacting leaks one. 2454 SENSITIVE_KEYS = ("password", "pass", "token", "secret", "key", "community") 2455 """Substrings marking a request parameter or AKiPS attribute whose value 2456 is redacted from log output. Extend it to redact more.""" 2457 2458 @classmethod 2459 def _is_sensitive_key(cls, name: str) -> bool: 2460 """Whether a parameter or attribute name looks like it holds a credential.""" 2461 return any(s in name.lower() for s in cls.SENSITIVE_KEYS) 2462 2463 def _redact_sensitive_params(self, params: dict[str, Any]) -> dict[str, Any]: 2464 """Return a copy of params with sensitive keys redacted from logging output.""" 2465 return { 2466 k: ("****" if self._is_sensitive_key(k) else v) for k, v in params.items() 2467 } 2468 2469 def _redact_text(self, text: str, literals: bool = True) -> str: 2470 """ 2471 Remove credentials from arbitrary text before it is logged or raised. 2472 2473 Matching on the query parameter covers the value whatever it looks 2474 like once URL encoded. Replacing the passwords this client holds 2475 catches them appearing outside a query string, but is unsafe for text 2476 that carries device data: a short password would rewrite every 2477 innocent occurrence of the same characters. Pass literals=False for 2478 anything that is not a URL or an error message. 2479 2480 Args: 2481 text (str): text that may contain credentials 2482 literals (bool): also replace the configured passwords wherever 2483 they appear (default: True) 2484 Returns: 2485 The text with any credential replaced by '****' 2486 """ 2487 text = re.sub(r"((?:password|passwd|pass)=)[^&\s]*", r"\1****", text) 2488 2489 # AKiPS keeps SNMP credentials as ordinary device attributes, so a 2490 # reply to something as innocent as get_device carries the community 2491 # string and the v3 auth and priv passwords. 2492 def redact_attribute(match: "re.Match[str]") -> str: 2493 if self._is_sensitive_key(match.group(2)): 2494 return f"{match.group(1)}****" 2495 return match.group(0) 2496 2497 text = re.sub(r"^(\S+\s\S+\s(\S+)\s=\s).*$", redact_attribute, text, flags=re.M) 2498 2499 if literals: 2500 for secret in (self.password, self.ro_password, self.rw_password): 2501 if secret: 2502 text = text.replace(secret, "****") 2503 return text 2504 2505 def _scrub_exception(self, err: BaseException) -> None: 2506 """ 2507 Strip credentials from an exception and everything it chains to. 2508 2509 AKiPS authenticates by query string and requests puts the failing URL 2510 in its exception messages, so an untouched exception carries the 2511 password into any log line or traceback that renders it. 2512 2513 The whole chain has to be scrubbed, not just the exception raised. 2514 requests raises its own error from the urllib3 one that caused it, and 2515 that inner exception holds the same URL, in its message and in a url 2516 attribute. Anything rendering a full traceback renders the chain, so 2517 leaving it means the password reaches wherever tracebacks are kept. 2518 2519 Rewriting in place keeps each exception's type and traceback, which a 2520 caller may be relying on, while making the text safe. 2521 2522 Args: 2523 err (BaseException): the exception to scrub, modified in place 2524 along with its __cause__ and __context__ chain 2525 """ 2526 seen: set[int] = set() 2527 pending: list[BaseException | None] = [err] 2528 while pending: 2529 node = pending.pop() 2530 if node is None or id(node) in seen: 2531 continue 2532 seen.add(id(node)) 2533 2534 original = str(node) 2535 redacted = self._redact_text(original) 2536 if redacted != original: 2537 node.args = (redacted,) 2538 2539 # urllib3 keeps the URL as an attribute of its own, which no 2540 # amount of message rewriting reaches 2541 url = getattr(node, "url", None) 2542 if isinstance(url, str): 2543 try: 2544 node.url = self._redact_text(url) # type: ignore[attr-defined] 2545 except AttributeError: 2546 pass 2547 2548 # A requests HTTPError carries the response it came from, and its 2549 # url is the one that was fetched, credentials and all. Error 2550 # reporters read that separately from the message. 2551 response = getattr(node, "response", None) 2552 if response is not None and isinstance(getattr(response, "url", None), str): 2553 try: 2554 response.url = self._redact_text(response.url) 2555 except AttributeError: 2556 pass 2557 2558 pending.append(node.__cause__) 2559 pending.append(node.__context__) 2560 2561 def _get( 2562 self, 2563 section: str = "api-db", 2564 params: dict[str, Any] | None = None, 2565 user: str | None = None, 2566 timeout: int | None = None, 2567 ) -> str: 2568 """ 2569 Base HTTP request against the AKiPS server for web API calls. 2570 2571 Sent as a POST with the password in the body, so it never appears in 2572 the request URI. Set use_post=False on the client to send the older 2573 GET form instead, which puts the password in the query string. The 2574 name is kept from when this only did GET, because it is called in 2575 two dozen places and is not part of the public API. 2576 2577 Section options are individually enabled via the AKiPS Web API Settings page. 2578 api-availability : Availability, default off 2579 api-db : Config and Events, default off 2580 api-config-viewer : Config Viewer, default off 2581 api-http-log : HTTP Log, default off 2582 api-flow : NetFlow, default off 2583 api-flow-timeseries : NetFlow Time-series, default off 2584 api-script : Site Script Functions, default off 2585 api-spm : Switch Port Mapper, default off 2586 api-msg : Syslog and Traps, default off 2587 api-unused-interfaces : Unused Interface, default off 2588 2589 Args: 2590 section (str): API section to call (default: 'api-db') 2591 params (dict): dictionary of parameters to pass to the server 2592 user (str): force the 'ro' or 'rw' account for this request 2593 timeout (int): seconds to wait for this one request, overriding 2594 the client's timeout. For a call whose cost does not depend 2595 on the client's usual work, such as a delete 2596 Returns: 2597 text output from the server 2598 Raises: 2599 AkipsCredentialError: if the account this section needs has no 2600 password 2601 AkipsAuthenticationError: if AKiPS rejects the credentials 2602 AkipsSectionDisabledError: if the section is not enabled on the 2603 server 2604 AkipsError: for any other error the AKiPS server returns 2605 requests.exceptions.HTTPError: for HTTP error responses 2606 requests.exceptions.ConnectionError: for connection errors 2607 requests.exceptions.Timeout: for request timeouts 2608 requests.exceptions.RequestException: for HTTP request errors 2609 """ 2610 server_url = f"https://{self.server}/{section}" 2611 2612 if section not in self.SECTION_USERS and section not in self._unknown_sections: 2613 # Warned rather than refused: AKiPS may add sections, and call() 2614 # exists so that reaching one does not have to wait for a release 2615 # here. A typo lands here too, which is the point. 2616 self._unknown_sections.add(section) 2617 logger.warning( 2618 "Unknown AKiPS API section {!r}, continuing anyway in case " 2619 "this server offers one this release does not know about. " 2620 "Known sections: {}".format( 2621 section, ", ".join(sorted(self.SECTION_USERS)) 2622 ) 2623 ) 2624 2625 # Work on a copy so credentials are never written into the dictionary 2626 # the caller passed in, and so params is optional as documented 2627 params = dict(params or {}) 2628 username, password = self._credentials_for(section, user) 2629 params["username"] = username 2630 2631 # The password travels in a POST body unless the caller has turned 2632 # that off, so it stays out of the request URI and out of everything 2633 # that records one. Everything else stays in the query string either 2634 # way, which is the form AKiPS documents and the only one older 2635 # servers accept. 2636 request_timeout = timeout if timeout is not None else self.timeout 2637 # A section may refuse the default method, so the choice is per 2638 # section rather than per client. See SECTION_METHODS. 2639 section_method = self.SECTION_METHODS.get(section, "POST") 2640 post = self.use_post and section_method == "POST" 2641 if self.use_post and not post and section not in self._method_warned: 2642 self._method_warned.add(section) 2643 # Logged at info, not warning. It is worth being able to see, 2644 # but an operator cannot act on it — the server is what refuses 2645 # the POST — and a permanent warning on every client teaches 2646 # people to ignore warnings, including the actionable ones this 2647 # module raises about a missing site script. 2648 logger.info( 2649 "Sending {} as {} rather than POST, so its password travels " 2650 "in the query string. {} does not answer a POST on any " 2651 "server seen so far. Set AKIPS.SECTION_METHODS[{!r}] = " 2652 "'POST' on a server where that is fixed.".format( 2653 section, section_method, section, section 2654 ) 2655 ) 2656 method = "POST" if post else "GET" 2657 data: dict[str, str] | None = None 2658 if post: 2659 data = {"password": password} 2660 else: 2661 params["password"] = password 2662 2663 logger.debug("{} url: {}".format(method, server_url)) 2664 logger.debug( 2665 "{} params: {}".format(method, self._redact_sensitive_params(params)) 2666 ) 2667 2668 try: 2669 with warnings.catch_warnings(): 2670 if not self.verify: 2671 # Scoped to this request on purpose. Disabling urllib3 2672 # warnings globally would also silence them for every 2673 # other library in the calling application. Note that 2674 # the warnings filter is process wide while this block 2675 # runs, so a concurrent thread could miss a warning. 2676 warnings.simplefilter( 2677 "ignore", urllib3.exceptions.InsecureRequestWarning 2678 ) 2679 if post: 2680 r = self.session.post( 2681 server_url, 2682 params=params, 2683 data=data, 2684 verify=self.verify, 2685 timeout=request_timeout, 2686 ) 2687 else: 2688 r = self.session.get( 2689 server_url, 2690 params=params, 2691 verify=self.verify, 2692 timeout=request_timeout, 2693 ) 2694 r.raise_for_status() 2695 except requests.exceptions.RequestException as err: 2696 # One handler for every requests failure: HTTPError, 2697 # ConnectionError and Timeout are all RequestException, and each 2698 # was doing the same thing here. The exception is scrubbed before 2699 # it is logged or re-raised, because requests reports the URL it 2700 # was fetching and AKiPS puts the password in that URL. 2701 self._scrub_exception(err) 2702 logger.error("AKiPS request failed: {}".format(err)) 2703 raise 2704 2705 # AKiPS can return a raw error message if something fails 2706 if re.match(r"^ERROR:", r.text): 2707 # Defense in depth: no AKiPS error seen so far echoes a credential 2708 # back, but this text goes into a log and an exception message. 2709 # Only the query parameter form is removed, never the password as 2710 # a literal, because a short one would rewrite matching characters 2711 # anywhere in the reply. 2712 message = self._redact_text(r.text, literals=False) 2713 logger.error("Web API request failed: {}".format(message)) 2714 # The two failures worth naming are the two that are nothing to do 2715 # with the call: the wrong password, and a section left switched 2716 # off. Both are ordinary first-run mistakes and both used to 2717 # arrive as an AkipsError saying only what AKiPS said. 2718 # 2719 # Matched loosely and on the distinctive phrase alone. AKiPS 2720 # prefixes the section name ('ERROR: api-db invalid 2721 # username/password') but that is not documented anywhere and 2722 # neither is the wording, so anything unrecognized has to keep 2723 # falling through to AkipsError rather than being forced into a 2724 # category. Both subclass AkipsError, so callers catching that 2725 # are unaffected. 2726 # Both carry what the call already knew, so a caller can act on 2727 # the section or the account without parsing AKiPS's prose. 2728 if re.search(r"invalid username/password", message, re.IGNORECASE): 2729 raise AkipsAuthenticationError( 2730 message=message, section=section, username=username 2731 ) 2732 if re.search(r"access is turned off", message, re.IGNORECASE): 2733 raise AkipsSectionDisabledError(message=message, section=section) 2734 raise AkipsError(message=message) 2735 else: 2736 logger.debug( 2737 "akips output: {}".format(self._redact_text(r.text, literals=False)) 2738 ) 2739 return r.text
logger =
<Logger akips (WARNING)>
class
AKIPS:
32class AKIPS: 33 """ 34 A class to handle interactions with the AKiPS Web API 35 36 AKiPS ships two API accounts, api-ro and api-rw, and its sections do not 37 all accept the same one. Supply the passwords for whichever accounts you 38 need and each call uses the right one; see SECTION_USERS below for the 39 mapping. A caller only reading data needs ro_password alone. 40 41 api = AKIPS('akips.example.com', ro_password='...', rw_password='...') 42 43 Four of the ten sections have methods of their own here: api-db, 44 api-script, api-msg and api-availability. The rest are reached through 45 call(), which sends a request to any section and parses the reply in the 46 same shapes those methods use. 47 48 AKiPS stores data in three levels, a parent such as a device or user, 49 then a child such as an interface or 'sys', then an attribute. What an 50 attribute's value means depends on its type, per the AKiPS API guide: 51 52 counter always 1, so the value carries nothing 53 enum '{integer},{text}', e.g. '2,down' 54 gauge a scale factor, positive to multiply and negative to 55 divide, not a reading 56 integer a whole number, positive, negative or zero 57 RTT microseconds, not milliseconds 58 text up to 2000 characters 59 timestamp seconds since the Unix epoch 60 uptime seconds since the status last changed 61 62 Counters and gauges therefore come back from get_attributes() as their 63 definition rather than a reading; the readings are in the time series 64 database, which get_latest_values() and get_series() read. 65 66 Time filters are not all the same shape either. 'lastNm' and 'lastNh' 67 are rolling windows, while 'lastNd' is calendar relative, so 'last1d' is 68 today rather than 24 hours. AKiPS will say which it means, since 'tf' is 69 one of the commands the read only account can run: 70 71 api.call('tf span last24h') 72 api.call('tf dump last1d') 73 74 """ 75 76 # Every API section AKiPS publishes, mapped to the account it accepts, as 77 # documented on the server's own Web API settings page. Every section 78 # takes api-ro except api-script, which requires api-rw, and api-db, which 79 # takes either: api-ro for read-only commands and api-rw for all of them. 80 # None marks that pair, where the read only account is preferred and a 81 # command needing more rights is reached with user='rw'. 82 # 83 # Only api-db reads a username at all. The others authenticate on the 84 # password alone, so the username sent alongside is ignored there. 85 # 86 # This doubles as the list of sections known to exist. Calling one that 87 # is not here is warned about rather than refused, because AKiPS may add 88 # sections and waiting for a release here would defeat the point of 89 # call(). Each section is also disabled by default on the server, so a 90 # section listed here can still be rejected until it is enabled. 91 SECTION_USERS: dict[str, str | None] = { 92 "api-availability": "api-ro", 93 "api-config-viewer": "api-ro", 94 "api-db": None, 95 "api-flow": "api-ro", 96 "api-flow-timeseries": "api-ro", 97 "api-http-log": "api-ro", 98 "api-msg": "api-ro", 99 "api-script": "api-rw", 100 "api-spm": "api-ro", 101 "api-unused-interfaces": "api-ro", 102 } 103 """Every API section AKiPS publishes, mapped to the account it accepts. 104 None marks a section taking either, where the read only account is 105 preferred. Also the list of sections known to exist.""" 106 107 SECTION_METHODS: dict[str, str] = { 108 "api-script": "GET", 109 } 110 """HTTP method to use per section, for the sections that cannot take the 111 default. Anything absent here is sent as POST when use_post is on. 112 113 **api-script does not answer a POST.** The server returns 200 headers in 114 about a quarter of a second, then sends no body and holds the connection 115 open until the client gives up, so every site script call hangs. The same 116 call as GET returns normally, and api-db takes a POST with the identical 117 header, so it is api-script specifically. Reported to AKiPS 2026-08-21. 118 119 **The cost is that those calls put the password back in the query 120 string**, which is what use_post exists to prevent. It applies to 121 get_device_by_ip(), set_group_membership() and delete_device(), and it is 122 api-rw for two of them. GET is not a preference: it is what the section 123 answers, and there is no third option, since the alternative is a call 124 that never returns. 125 126 Nothing here assumes that will change. Sending the password in a POST 127 body is itself undocumented — AKiPS support gave it out rather than the 128 API guide describing it — so what any given server accepts is a question 129 for that server rather than something this module can predict. This is a 130 class attribute for that reason: if a server does take a POST on a 131 section, say so without waiting for a release here. 132 133 AKIPS.SECTION_METHODS["api-script"] = "POST" 134 135 That is class wide and affects every client in the process.""" 136 137 def __init__( 138 self, 139 server: str, 140 username: str = "api-ro", 141 password: str | None = None, 142 verify: bool | str = True, 143 timezone: str = "America/New_York", 144 timeout: int = 30, 145 ro_password: str | None = None, 146 rw_password: str | None = None, 147 use_post: bool = True, 148 ) -> None: 149 self.server = server 150 """The AKiPS server hostname or IP address.""" 151 self.username = username 152 """Kept for callers who set it directly. With 'api-ro' or 'api-rw' 153 the password given alongside fills that account; with any other name 154 that pair is used for every section, which is how to use a custom 155 AKiPS API account.""" 156 self.password = password 157 """The password paired with username.""" 158 self.ro_password = ro_password 159 """Password for the api-ro account.""" 160 self.rw_password = rw_password 161 """Password for the api-rw account.""" 162 self.verify = verify 163 """Whether to verify TLS certificates. A path to a CA bundle can be 164 given instead, which is how to trust a server whose chain is missing 165 an intermediate without turning verification off entirely.""" 166 self.server_timezone = timezone 167 """Timezone of the AKiPS server, used to read the epochs it sends.""" 168 self.timeout = timeout 169 """HTTP timeout in seconds applied to every call. Assign to it to 170 change the timeout of an existing client, e.g. api.timeout = 60.""" 171 self.use_post = use_post 172 """Whether to send the password in a POST body instead of the query 173 string. True by default, and it should stay that way: URLs are 174 recorded by web servers, proxies and load balancers in their access 175 logs, and appear in exception messages and client history. A request 176 body is not logged that way, and a credential does not belong in a URL. 177 178 Set it to False only for a server that will not accept the POST form, 179 which puts the password back in the URL. Nothing falls back on its 180 own, because a silent retry over GET would leak the password at 181 exactly the moment the server turned out not to support this.""" 182 self.session = requests.Session() 183 """The requests session every call is made through.""" 184 # Sections warned about already, so a caller legitimately using a 185 # section this release does not know about is told once rather 186 # than on every call 187 self._unknown_sections: set[str] = set() 188 # Sections already warned about for falling back to GET, so a poll 189 # loop is told once rather than on every call 190 self._method_warned: set[str] = set() 191 192 # A username other than the two built in accounts is used for every 193 # section. AKiPS does not offer custom API accounts yet, but this is 194 # where they will land, and it keeps working for anyone already 195 # passing username and password directly. 196 self._account_override: tuple[str, str] | None = None 197 if password is not None: 198 if username == "api-ro" and self.ro_password is None: 199 self.ro_password = password 200 elif username == "api-rw" and self.rw_password is None: 201 self.rw_password = password 202 elif username not in ("api-ro", "api-rw"): 203 self._account_override = (username, password) 204 205 if ( 206 self._account_override is None 207 and self.ro_password is None 208 and self.rw_password is None 209 ): 210 raise AkipsCredentialError( 211 "No AKiPS password provided. Pass ro_password, rw_password, " 212 "or a username and password pair." 213 ) 214 215 # --------------------------------------------------------------------------- 216 # api-db interface methods, these use the 'api-ro' or 'api-rw' user 217 218 # entities commands 219 220 def get_devices( 221 self, group_filter: str = "any", groups: list[str] | None = None 222 ) -> dict[str, dict[str, str | None]] | None: 223 """ 224 Pull a list of all devices and six key attributes of each, optionally 225 filtered by group membership. 226 227 This reads the 'sys' child and nothing else, and asks it for six 228 attributes: ip4addr, SNMPv2-MIB.sysName, SNMPv2-MIB.sysDescr, 229 SNMPv2-MIB.sysObjectID, SNMPv2-MIB.sysLocation and 230 SNMPv2-MIB.sysContact. Those are the values the AKiPS device edit page 231 shows read only, being what SNMP reported rather than what an operator 232 set, plus the address. 233 234 Both the child and the six are fixed here rather than arguments, 235 because this is the inventory view: every device comes back carrying 236 all six, as None where it reported no value, so they can be listed or 237 tabulated without checking each key first. Anything else the server 238 returns for a device is kept alongside them rather than dropped. 239 240 sysObjectID is worth knowing about: it identifies the model, such as 241 'ARUBA-MIB.ap225', which is often the field an inventory actually wants 242 and is more reliably populated than sysLocation. 243 244 For other attributes, other children, or a device's whole contents, 245 see get_attributes() and get_device(). 246 247 Because it asks for one child, this is the only method returning 248 attributes that does not keep the child level; the result is flattened 249 to device and attribute, which is the shape a listing wants. Should a 250 reply ever carry more than one child, their attributes are merged and 251 the last one read wins, where get_attributes() would keep them apart. 252 253 Supporting AKiPS command syntax: 254 255 mget {type} [{parent regex} [{child regex} [{attribute regex}]]] 256 [descr {/regex/}] [value {text|integer|/regex/}] 257 [profile {profile name}] [any|all|not group {group name} ...] 258 259 Args: 260 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 261 groups (list): list of group names to filter by (if any) 262 Returns: 263 A dictionary of device names to attribute dictionaries, or None if no devices found 264 Raises: 265 AkipsError: if the AKiPS server returns an error 266 """ 267 # The polled values the AKiPS device edit page shows read only, plus 268 # the address. Keeping to that set is deliberate: they are the 269 # standard SNMP system group fields an operator already recognizes. 270 attributes = [ 271 "ip4addr", 272 "SNMPv2-MIB.sysName", 273 "SNMPv2-MIB.sysDescr", 274 "SNMPv2-MIB.sysObjectID", 275 "SNMPv2-MIB.sysLocation", 276 "SNMPv2-MIB.sysContact", 277 ] 278 cmd_attributes = "|".join(attributes) 279 params = { 280 "cmds": f"mget text * sys /{cmd_attributes}/", 281 } 282 if groups: 283 # [any|all|not group {group name} ...] 284 group_list = " ".join(groups) 285 params["cmds"] += f" {group_filter} group {group_list}" 286 text = self._get(params=params) 287 if text: 288 data: dict[str, dict[str, str | None]] = {} 289 for parent, children in self._parse_attributes(text).items(): 290 # Every requested attribute is present, as None where the 291 # device reported no value for it 292 entry: dict[str, str | None] = dict.fromkeys(attributes) 293 for child_attributes in children.values(): 294 entry.update(child_attributes) 295 data[parent] = entry 296 # A reply that parses to nothing is nothing found, the same 297 # answer an empty reply gives, rather than an empty container 298 if not data: 299 return None 300 logger.debug("Found {} devices in akips".format(len(data.keys()))) 301 return data 302 return None 303 304 def get_device(self, device: str) -> dict[str, dict[str, str | None]] | None: 305 """ 306 Pull all configuration attributes for a single device. The name is the 307 device's AKiPS name, its one primary key, which is either its sysName 308 or its IP address depending on how the server is set to name devices. 309 It is assigned at discovery, but a server can be told to reassign 310 devices already discovered from the other source, and an operator can 311 change one by hand, so a caller storing these as identifiers of its own 312 should not assume they never change. A device keyed by name still 313 carries its address as an attribute, and get_device_by_ip() resolves 314 an address back to the key. 315 316 This is the deep dive: every child and attribute this device holds, 317 which varies by device type. For the same fields across every device, 318 see get_devices(). 319 320 The result is the one device's children and their attributes, not a 321 dictionary keyed by the name that was just passed in: 322 323 device = api.get_device('TH840-A') 324 device['sys']['ip4addr'] 325 326 get_attributes() is the same query without that assumption, and keys 327 its result by device because it can match several. 328 329 Supporting AKiPS command syntax: 330 331 mget {type} [{parent regex} [{child regex} [{attribute regex}]]] 332 [descr {/regex/}] [value {text|integer|/regex/}] 333 [profile {profile name}] [any|all|not group {group name} ...] 334 335 Args: 336 device (str): the AKiPS name of one device, exactly. AKiPS matches 337 a bare name in this position exactly, so this is unambiguous; a 338 '/regex/' is refused, since matching several devices is what 339 get_attributes() is for 340 Returns: 341 A dictionary of the device's child names to attribute names and 342 values, or None if the device was not found 343 Raises: 344 ValueError: if a pattern is given, or if the reply somehow held 345 more than one device 346 AkipsError: if the AKiPS server returns an error 347 """ 348 # A bare name matches exactly in the parent position, but a '/regex/' 349 # does not, and this method has nowhere to put a second device. 350 if device.startswith("/") and device.endswith("/") and len(device) > 1: 351 raise ValueError( 352 "get_device takes one device name, not a pattern. Use " 353 "get_attributes(device={!r}) to match several".format(device) 354 ) 355 # get_attributes() with the filters left at their defaults, rather than 356 # building the same command a second time. 357 data = self.get_attributes(device=device) 358 if not data: 359 return None 360 if len(data) > 1: 361 raise ValueError( 362 "get_device matched {} devices ({}). Use get_attributes() " 363 "for more than one".format(len(data), ", ".join(sorted(data))) 364 ) 365 return next(iter(data.values())) 366 367 # The children ping and SNMP state are reported under. Naming them saves 368 # AKiPS walking every child of every device, which is most of the cost of 369 # this query: on a 16,000 device fleet the wildcard took 10.5s against 370 # 5.0s here, for the same rows. ping6 is listed though most sites monitor 371 # over IPv4 alone, because a device reachable only over IPv6 going down is 372 # exactly what this call must not miss, and an alternative that matches 373 # nothing costs nothing. 374 UNREACHABLE_CHILDREN = "ping4|ping6|sys" 375 """The children get_unreachable() searches, as a regex.""" 376 377 def get_unreachable( 378 self, children: str = UNREACHABLE_CHILDREN 379 ) -> dict[str, dict[str, Any]] | None: 380 """ 381 Pull a list of unreachable devices by Ping and SNMP state. 382 383 Supporting AKiPS command syntax: 384 385 mget {type} [{parent regex} [{child regex} [{attribute regex}]]] 386 [descr {/regex/}] [value {text|integer|/regex/}] 387 [profile {profile name}] [any|all|not group {group name} ...] 388 389 Note the {type} field is left off here. These are enum attributes, so 390 narrowing with 'mget text' returns no rows at all rather than an 391 error, even though it is the obvious thing to reach for. 392 393 **Asking only for what is broken is not necessarily the cheap way.** 394 On one fleet this call took roughly three times as long as 395 get_ping_state(), while returning 289 times fewer rows: filtering on 396 value makes AKiPS match every device's enum rather than dump the 397 attribute, and this asks for two attributes where that asks for one. 398 Whether that holds elsewhere is unknown, and it may be a property of 399 that server's data rather than of the query. It is recorded because 400 the opposite is the natural assumption: if this call is hot, measure 401 it against reading the state outright and filtering here. 402 403 Args: 404 children (str): regex of children to search, defaulting to the 405 ones AKiPS reports these attributes under. Pass '*' for 406 every child of every device, which is correct for a site 407 naming them differently and considerably slower 408 Returns: 409 A dictionary of device names to their unreachable attributes, or 410 None if nothing was reported as down 411 Raises: 412 AkipsError: if the AKiPS server returns an error 413 """ 414 # '*' is the wildcard rather than a pattern, so it is the one value 415 # that must not be wrapped in slashes 416 child = children if children == "*" else f"/{children}/" 417 params = { 418 "cmds": f"mget * * {child} /PING.icmpState|SNMP.snmpState/ value /down/", 419 } 420 text = self._get(params=params) 421 if text: 422 data: dict[str, dict[str, Any]] = {} 423 unparsed = [] 424 lines = text.split("\n") 425 for line in lines: 426 match = re.match( 427 r"^(\S+)\s(\S+)\s(\S+)\s=\s(\S+),(\S+),(\S+),(\S+),(\S+)?$", line 428 ) 429 if not match: 430 if line.strip(): 431 # A line reporting a device down that this does not 432 # understand must not vanish: under reporting an 433 # outage is the worst thing this call can do. 434 unparsed.append(line) 435 continue 436 # epoch fields are in the server's timezone 437 name = match.group(1) 438 attribute = match.group(3) 439 event_start = datetime.fromtimestamp( 440 int(match.group(7)), tz=pytz.timezone(self.server_timezone) 441 ) 442 device_added = datetime.fromtimestamp( 443 int(match.group(6)), tz=pytz.timezone(self.server_timezone) 444 ) 445 if name not in data: 446 # populate a starting point for this device 447 data[name] = { 448 "name": name, 449 "ping_state": "n/a", 450 "snmp_state": "n/a", 451 "event_start": event_start, # epoch in local timezone 452 } 453 if attribute == "PING.icmpState": 454 data[name]["ping_state"] = match.group(5) 455 # A device down on both checks reports one child, index 456 # and address. Ping wins them, because it is the only 457 # line carrying an address, and assigning here while the 458 # SNMP branch below only fills gaps makes the result the 459 # same whichever order the lines arrive in. 460 data[name]["child"] = match.group(2) 461 data[name]["index"] = match.group(4) 462 data[name]["device_added"] = device_added 463 data[name]["ip4addr"] = match.group(8) 464 elif attribute == "SNMP.snmpState": 465 data[name]["snmp_state"] = match.group(5) 466 data[name].setdefault("child", match.group(2)) 467 data[name].setdefault("index", match.group(4)) 468 data[name].setdefault("device_added", device_added) 469 data[name].setdefault("ip4addr", None) 470 # A device down on both ping and SNMP has two start times; the 471 # outage began at the earlier of them. This has to be the only 472 # place event_start is set, or the comparison is against the 473 # value just written from this same line and the last line seen 474 # would always win. 475 if event_start < data[name]["event_start"]: 476 data[name]["event_start"] = event_start 477 if unparsed: 478 logger.warning( 479 "Could not parse {} of {} unreachable lines from akips, " 480 "those devices are missing from the result. First: {}".format( 481 len(unparsed), len(unparsed) + len(data), unparsed[0][:200] 482 ) 483 ) 484 # A reply that parses to nothing is nothing found, the same 485 # answer an empty reply gives, rather than an empty container 486 if not data: 487 return None 488 logger.debug("Found {} devices in akips".format(len(data))) 489 return data 490 return None 491 492 def get_ping_state( 493 self, 494 states: tuple[str, ...] | list[str] | None = None, 495 child: str = "ping4", 496 group_filter: str = "any", 497 groups: list[str] | None = None, 498 ) -> dict[str, dict[str, Any]] | None: 499 """ 500 Pull the ping state of every device, with when it last changed. 501 502 get_unreachable() answers 'what is broken now' and so asks only for 503 the devices that are down. This asks the same record without that 504 filter, which is what to call for a device that is up: its state, and 505 both of the epochs the enum carries. 506 507 **The two epochs are the useful part and their names undersell them.** 508 'created' is when AKiPS started polling the device, so it is the date 509 the device was added to AKiPS, and it is not otherwise reachable for a 510 device that is healthy. 'modified' is the instant the state last 511 changed, not a row-touched timestamp. Both arrive as aware datetimes 512 in the server's timezone rather than as the integers the raw attribute 513 holds, so nothing needs converting. 514 515 Those two answer the column AKiPS shows on its own device dashboard, 516 the one reading Uptime on a device that is up and Downtime on a device 517 that is down. It is not sysUpTime: the figure is now minus 'modified' 518 and 'value' decides which word. sysUpTime counts from the last boot 519 and keeps counting through an outage, so the two disagree on exactly 520 the devices somebody is looking at. 521 522 get_snmp_state() is the same record for the SNMP agent. It answers 523 for far fewer devices, because AKiPS pings everything it holds and 524 polls SNMP only where SNMP is configured. 525 526 Args: 527 states (list): only report these states, e.g. ('down',), or None 528 for every device whatever its state, which is the default 529 child (str): which ping child to read (default: 'ping4'). Pass 530 'ping4|ping6' for both, though a device answering on both 531 keeps only one entry and warns, since this is keyed by device 532 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 533 groups (list): list of group names to filter by (if any) 534 Returns: 535 A dictionary of device names to the parsed state, or None if no 536 device matched. Each entry carries the enum fields described on 537 _parse_enum, plus the device 'name' and 'child' 538 Raises: 539 AkipsError: if the AKiPS server returns an error 540 """ 541 return self._get_enum_attribute( 542 "PING.icmpState", 543 child=child, 544 values=states, 545 group_filter=group_filter, 546 groups=groups, 547 ) 548 549 def get_snmp_state( 550 self, 551 states: tuple[str, ...] | list[str] | None = None, 552 child: str = "sys", 553 group_filter: str = "any", 554 groups: list[str] | None = None, 555 ) -> dict[str, dict[str, Any]] | None: 556 """ 557 Pull the SNMP agent state of every SNMP polled device. 558 559 The SNMP counterpart to get_ping_state(), reading the same kind of 560 record with the same fields: the state, when the device was added, 561 and when the state last changed. 562 563 **It answers for fewer devices than get_ping_state() does, and the 564 difference is large.** AKiPS pings everything it holds but polls SNMP 565 only where SNMP is configured, so a device absent from this result is 566 usually one that is not SNMP polled rather than one whose agent has 567 stopped answering. Measured on one fleet, 5,821 devices of 16,785 568 appeared here and the rest were ICMP only. A caller that assumes the 569 same denominator as the ping call reads two thirds of the fleet as 570 broken. 571 572 Absence therefore means unknown, not down. The devices answering here 573 are the same ones answering SNMPv2-MIB.sysUpTime, which is a way to 574 confirm the denominator on a given server. 575 576 Args: 577 states (list): only report these states, e.g. ('down',), or None 578 for every SNMP polled device, which is the default 579 child (str): which child holds the agent state (default: 'sys') 580 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 581 groups (list): list of group names to filter by (if any) 582 Returns: 583 A dictionary of device names to the parsed state, or None if no 584 device matched. Each entry carries the enum fields described on 585 _parse_enum, plus the device 'name' and 'child' 586 Raises: 587 AkipsError: if the AKiPS server returns an error 588 """ 589 return self._get_enum_attribute( 590 "SNMP.snmpState", 591 child=child, 592 values=states, 593 group_filter=group_filter, 594 groups=groups, 595 ) 596 597 def get_attributes( 598 self, 599 device: str = "*", 600 child: str = "*", 601 attribute: str = "*", 602 value: str | None = None, 603 group_filter: str = "any", 604 groups: list[str] | None = None, 605 ) -> dict[str, dict[str, dict[str, str | None]]] | None: 606 """ 607 Pull attribute values with variable search criteria. Search criteria defaults to 608 a wildcard match but can be filtered by 'device' name or pattern, 'child' name or pattern, 609 'attribute' name or pattern, and/or attribute 'value' or pattern. Additionally, 610 results can be filtered by group membership using 'any', 'all', or 'not' operators 611 along with one or more group names. 612 613 Supporting AKiPS command syntax: 614 615 mget {type} [{parent regex} [{child regex} [{attribute regex}]]] 616 [descr {/regex/}] [value {text|integer|/regex/}] 617 [profile {profile name}] [any|all|not group {group name} ...] 618 619 **A bare attribute name matches exactly and matches nothing.** AKiPS 620 qualifies most attributes with their MIB, so 'sysUpTime' matches no 621 attribute anywhere while 'SNMPv2-MIB.sysUpTime' matches on every 622 device that reports it. The unqualified form is not an error: it 623 returns an empty result, which looks exactly like a fleet where 624 nothing reports that attribute. Use a pattern, '/sysUpTime/', when 625 the qualified name is not known. 626 627 Args: 628 device (str): device name or pattern to match (default: '*') 629 child (str): child name or pattern to match (default: '*') 630 attribute (str): attribute name or pattern to match (default: '*'). 631 A bare name must match exactly, MIB prefix included; see above 632 value (str): value or pattern to match (default: None) 633 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 634 groups (list): list of group names to filter by (if any) 635 Returns: 636 A nested dictionary of device names to child names to attribute names and values, 637 or None if no devices found 638 Raises: 639 AkipsError: if the AKiPS server returns an error 640 """ 641 params = { 642 "cmds": f"mget * {device} {child} {attribute}", 643 } 644 if value: 645 # [value {text|/regex/|integer|ipaddr}] 646 params["cmds"] += f" value {value}" 647 if groups: 648 # [any|all|not group {group name} ...] 649 group_list = " ".join(groups) 650 params["cmds"] += f" {group_filter} group {group_list}" 651 text = self._get(params=params) 652 if text: 653 data = self._parse_attributes(text) 654 # A reply that parses to nothing is nothing found, the same 655 # answer an empty reply gives, rather than an empty container 656 if not data: 657 return None 658 logger.debug("Found {} devices in akips".format(len(data.keys()))) 659 return data 660 return None 661 662 # UPS helpers 663 664 # UPS output sources other than 'normal'. A UPS reporting any of these is 665 # not running on mains, which is what an operator wants to know about. 666 # 667 # In MIB order, since UPS-MIB numbers them other(1) none(2) normal(3) 668 # bypass(4) battery(5) booster(6) reducer(7). 'none' is a UPS delivering 669 # no output at all and 'other' one that cannot classify its own source; 670 # both are here for the same reason 'unknown' is in the battery states 671 # below, because a UPS that cannot answer the question is worth looking 672 # at too. 673 UPS_ABNORMAL_OUTPUT_SOURCES = ( 674 "other", 675 "none", 676 "bypass", 677 "battery", 678 "booster", 679 "reducer", 680 ) 681 """Output sources get_ups_output_source() reports by default, being every 682 UPS-MIB source but normal, so every state that is not running on mains.""" 683 684 # Battery states other than batteryNormal. 'unknown' is included because 685 # a UPS that cannot report its own battery is worth looking at too. 686 UPS_ABNORMAL_BATTERY_STATES = ("unknown", "batteryLow", "batteryDepleted") 687 """Battery states get_ups_battery_status() reports by default, being 688 every UPS-MIB state but batteryNormal.""" 689 690 # The attribute Liebert and Vertiv equipment reports battery test results 691 # in. Battery test results are not in the standard UPS-MIB, so every 692 # vendor uses its own; this one is named in the method that reads it. 693 LIEBERT_BATTERY_TEST_ATTRIBUTE = "LIEBERT-GP-POWER-MIB.lgpPwrBatteryTestResult" 694 """The attribute get_liebert_battery_test() reads. Battery test results 695 are not in the standard UPS-MIB, so this one is vendor specific.""" 696 697 def get_ups_battery_status( 698 self, 699 states: tuple[str, ...] | list[str] | None = UPS_ABNORMAL_BATTERY_STATES, 700 group_filter: str = "any", 701 groups: list[str] | None = None, 702 ) -> dict[str, dict[str, Any]] | None: 703 """ 704 Pull the UPS devices whose battery is not reporting as normal. 705 706 UPS-MIB reports the battery's own condition, separately from where the 707 UPS is drawing its output, which get_ups_output_source() reads. By 708 default this returns only the states other than batteryNormal. 709 710 This is the battery's condition, not how long it would last. AKiPS 711 keeps the numeric readings such as upsEstimatedMinutesRemaining in its 712 time series database rather than alongside these, so they come from 713 get_series() rather than from here. Reading them with mget returns 714 the gauge's scaling factor, which is identical for every device. 715 716 Args: 717 states (list): battery states to report, defaulting to everything 718 except batteryNormal. Pass None for every UPS whatever its 719 battery state 720 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 721 groups (list): list of group names to filter by (if any) 722 Returns: 723 A dictionary of device names to the parsed state, or None if no 724 device matched. Each entry carries the enum fields described on 725 _parse_enum, where 'value' is the battery state and 'modified' is 726 when it last changed, plus the device 'name' and 'child' 727 Raises: 728 AkipsError: if the AKiPS server returns an error 729 """ 730 return self._get_enum_attribute( 731 "UPS-MIB.upsBatteryStatus", 732 child="battery", 733 values=states, 734 group_filter=group_filter, 735 groups=groups, 736 ) 737 738 def get_ups_output_source( 739 self, 740 states: tuple[str, ...] | list[str] | None = UPS_ABNORMAL_OUTPUT_SOURCES, 741 group_filter: str = "any", 742 groups: list[str] | None = None, 743 ) -> dict[str, dict[str, Any]] | None: 744 """ 745 Pull the UPS devices that are not running on mains power. 746 747 UPS-MIB reports where a UPS is drawing its output from, which is 748 'normal' when all is well. By default this returns every other value, 749 so the result is the list of UPSes worth looking at. That includes 750 'none', a UPS delivering no output at all, and 'other', one that 751 cannot classify its own source. 752 753 Note this is the output source, not the battery's own health, which 754 UPS-MIB reports separately as upsBatteryStatus. 755 756 Args: 757 states (list): output sources to report, defaulting to everything 758 except 'normal'. Pass None for every UPS whatever its state 759 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 760 groups (list): list of group names to filter by (if any) 761 Returns: 762 A dictionary of device names to the parsed state, or None if no 763 device matched. Each entry carries the enum fields described on 764 _parse_enum, where 'value' is the output source and 'modified' is 765 when it last changed, plus the device 'name' and 'child' 766 Raises: 767 AkipsError: if the AKiPS server returns an error 768 """ 769 return self._get_enum_attribute( 770 "UPS-MIB.upsOutputSource", 771 # The child the UPS itself is reported under, as against 772 # 'battery' for the battery attributes. Naming it keeps AKiPS 773 # from walking every child of every device, which is most of the 774 # cost of the query. 775 child="ups", 776 values=states, 777 group_filter=group_filter, 778 groups=groups, 779 ) 780 781 def get_liebert_battery_test( 782 self, 783 results: tuple[str, ...] | list[str] | None = ("failed",), 784 attribute: str = LIEBERT_BATTERY_TEST_ATTRIBUTE, 785 group_filter: str = "any", 786 groups: list[str] | None = None, 787 ) -> dict[str, dict[str, Any]] | None: 788 """ 789 Pull the results of the last battery self test on Liebert and Vertiv 790 UPS equipment. 791 792 By default this returns only the failures, which is the list of 793 batteries to replace. Pass results=None for every UPS and its last 794 result. 795 796 The vendor is in the name on purpose. Battery test results are not in 797 the standard UPS-MIB, so this reads an attribute only Liebert and 798 Vertiv equipment reports. Run against another vendor's fleet it 799 returns nothing, which would otherwise read as good news. Another 800 vendor's equivalent attribute can be passed to reuse the same parsing 801 and shape. 802 803 Args: 804 results (list): test results to report, defaulting to failures 805 only. Pass None for every UPS whatever its last result 806 attribute (str): the vendor attribute holding the result 807 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 808 groups (list): list of group names to filter by (if any) 809 Returns: 810 A dictionary of device names to the parsed result, or None if no 811 device matched. Each entry carries the enum fields described on 812 _parse_enum, where 'value' is the test result and 'modified' is 813 when it last changed, plus the device 'name' and 'child' 814 Raises: 815 AkipsError: if the AKiPS server returns an error 816 """ 817 return self._get_enum_attribute( 818 attribute, 819 child="battery", 820 values=results, 821 group_filter=group_filter, 822 groups=groups, 823 ) 824 825 # group commands 826 827 def get_group_membership( 828 self, 829 device: str = "*", 830 group_filter: str = "any", 831 groups: list[str] | None = None, 832 ) -> dict[str, list[str]] | None: 833 """ 834 Pull a list of device names to group memberships. Defaults to all devices 835 and all groups (including the special 'maintenance_mode' group). 836 837 Supporting AKiPS command syntax: 838 839 mgroup {type} [{parent regex}] 840 [any|all|not group {group name} ...] 841 842 Args: 843 device (str): device name or pattern to match (default: '*') 844 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 845 groups (list): list of group names to filter by (if any) 846 Returns: 847 A dictionary of device names to lists of group names, or None if no devices found 848 Raises: 849 AkipsError: if the AKiPS server returns an error 850 """ 851 params = { 852 "cmds": f"mgroup * {device}", 853 } 854 if groups: 855 group_list = " ".join(groups) 856 params["cmds"] += f" {group_filter} group {group_list}" 857 text = self._get(params=params) 858 if text: 859 data = { 860 device_name: groups_value.split(",") 861 for device_name, groups_value in self._parse_key_value(text).items() 862 } 863 # A reply that parses to nothing is nothing found, the same 864 # answer an empty reply gives, rather than an empty container 865 if not data: 866 return None 867 logger.debug( 868 "Found {} device and group mappings in akips".format(len(data.keys())) 869 ) 870 return data 871 return None 872 873 # event commands 874 875 def get_events( 876 self, 877 event_type: str = "all", 878 period: str = "last1h", 879 device: str = "*", 880 child: str = "*", 881 attribute: str = "*", 882 group_filter: str = "any", 883 groups: list[str] | None = None, 884 ) -> list[dict[str, str]] | None: 885 """ 886 Pull a list of events over a time period with optional filtering by device, 887 child, attribute, and/or group membership. Defaults to all event types over 888 the last hour. Review AKiPS documentation for details on event types and 889 time filter syntax. 890 891 Supporting AKiPS command syntax: 892 893 mget event {all,critical,enum,threshold,uptime} 894 time {time filter} [{parent regex} {child regex} 895 {attribute regex}] [profile {profile name}] 896 [any|all|not group {group name} ...] 897 898 Args: 899 event_type (str): type of events to retrieve (default: 'all') 900 period (str): time period to retrieve events from (default: 'last1h') 901 device (str): device name or pattern to match (default: '*') 902 child (str): child name or pattern to match (default: '*') 903 attribute (str): attribute name or pattern to match (default: '*') 904 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 905 groups (list): list of group names to filter by (if any) 906 Returns: 907 A list of event dictionaries, or None if no events found 908 Raises: 909 AkipsError: if the AKiPS server returns an error 910 """ 911 params = { 912 "cmds": f"mget event {event_type} time {period} {device} {child} {attribute}" 913 } 914 if groups: 915 # [any|all|not group {group name} ...] 916 group_list = " ".join(groups) 917 params["cmds"] += f" {group_filter} group {group_list}" 918 text = self._get(params=params) 919 if text: 920 data = [] 921 lines = text.split("\n") 922 for line in lines: 923 match = re.match( 924 r"^(\S+)\s(\S+)\s(\S+)\s(\S+)\s(\S+)\s(\S+)\s(.*)$", line 925 ) 926 if match: 927 entry = { 928 "epoch": match.group(1), 929 "parent": match.group(2), 930 "child": match.group(3), 931 "attribute": match.group(4), 932 "type": match.group(5), 933 "flags": match.group(6), 934 "details": match.group(7), 935 } 936 data.append(entry) 937 # A reply that parses to nothing is nothing found, the same 938 # answer an empty reply gives, rather than an empty container 939 if not data: 940 return None 941 logger.debug( 942 "Found {} events of type {} in akips".format(len(data), event_type) 943 ) 944 return data 945 return None 946 947 # time series commands 948 949 def get_series( 950 self, 951 period: str = "last1h", 952 time_interval: int = 60, 953 device: str = "*", 954 attribute: str = "*", 955 get_dict: bool = True, 956 group_filter: str = "any", 957 groups: list[str] | None = None, 958 ) -> list[dict[str, str]] | list[list[str]] | None: 959 """ 960 Pull a series of counter values with average values over a time period with optional 961 filtering by device, attribute, and/or group membership. Defaults to all devices 962 and attributes over the last hour with 60 second intervals. Review AKiPS documentation 963 for details on time filter syntax. 964 965 Supporting AKiPS command syntax: 966 967 cseries [interval total|avg {secs}] time {time filter} 968 {type} {parent regex} {child regex} {attribute regex} 969 [profile {profile name}] [any|all|not group {group name} ...] 970 971 Args: 972 period (str): time period to retrieve series from (default: 'last1h') 973 time_interval (int): interval in seconds for series data points (default: 60) 974 device (str): device name or pattern to match (default: '*') 975 attribute (str): attribute name or pattern to match (default: '*') 976 get_dict (bool): return each row as a dictionary keyed by the 977 header row, rather than the CSV as sent (default: True). 978 The header carries one column heading per interval, which is 979 the time axis for the values under it. As dictionaries those 980 headings are the keys; as lists the header is the first entry, 981 so the list form has one row more than the dictionary form 982 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 983 groups (list): list of group names to filter by (if any) 984 Returns: 985 A list of series data rows (as dictionaries or lists), or None if no data found 986 Raises: 987 AkipsError: if the AKiPS server returns an error 988 """ 989 params = { 990 "cmds": f"cseries interval avg {time_interval} time {period} * {device} * {attribute}" 991 } 992 if groups: 993 group_list = " ".join(groups) 994 params["cmds"] += f" {group_filter} group {group_list}" 995 text = self._get(params=params) 996 if text: 997 # Rows as dictionaries keyed by the header row, or as the CSV as 998 # sent with that header kept as the first entry. The header is 999 # the time axis, one column heading per interval, so the list form 1000 # keeps it: without it the values underneath are readings with no 1001 # timestamps. The dictionary form does not need it separately 1002 # because those headings became its keys. 1003 csv_to_list = self._parse_csv(text, header=get_dict) 1004 data_rows = csv_to_list if get_dict else csv_to_list[1:] 1005 if not data_rows: 1006 # A header with nothing under it is an axis with no series on 1007 # it, which is nothing found rather than a result 1008 return None 1009 logger.debug("Found {} series entries".format(len(csv_to_list))) 1010 return csv_to_list 1011 return None 1012 1013 def get_latest_values( 1014 self, 1015 attribute: str, 1016 device: str = "*", 1017 child: str = "*", 1018 period: str = "last1h", 1019 time_interval: int = 300, 1020 group_filter: str = "any", 1021 groups: list[str] | None = None, 1022 ) -> dict[str, dict[str, dict[str, Any]]] | None: 1023 """ 1024 Pull the most recent reading of a numeric attribute for each device. 1025 1026 Numeric attributes do not hold a reading in the config database that 1027 get_attributes() reads; that holds the counter or gauge definition, 1028 which is the same for every device. The readings live in the time 1029 series database, so this asks for a short series and keeps the last 1030 value in it. 1031 1032 The final interval of a series is usually still being filled and comes 1033 back empty, so the last column is not the answer; this returns the 1034 last column that has a value, along with when it was measured. Values 1035 are already scaled by AKiPS, so what comes back is in the attribute's 1036 real units. 1037 1038 Supporting AKiPS command syntax: 1039 1040 cseries [interval total|avg {secs}] time {time filter} 1041 {type} {parent regex} {child regex} {attribute regex} 1042 [profile {profile name}] [any|all|not group {group name} ...] 1043 1044 Args: 1045 attribute (str): the attribute to read 1046 device (str): device name or pattern to match (default: '*') 1047 child (str): child name or pattern to match (default: '*') 1048 period (str): how far back to look (default: 'last1h'). It only 1049 has to be long enough to contain one completed interval 1050 time_interval (int): seconds per interval (default: 300) 1051 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 1052 groups (list): list of group names to filter by (if any) 1053 Returns: 1054 A dictionary of device names to child names to the reading, each 1055 with 'value', 'time' and 'attribute'. A device with no reading in 1056 the period is present with a value of None rather than dropped. 1057 None if nothing matched at all 1058 Raises: 1059 AkipsError: if the AKiPS server returns an error 1060 """ 1061 params = { 1062 "cmds": f"cseries interval avg {time_interval} time {period} " 1063 f"* {device} {child} {attribute}" 1064 } 1065 if groups: 1066 # [any|all|not group {group name} ...] 1067 group_list = " ".join(groups) 1068 params["cmds"] += f" {group_filter} group {group_list}" 1069 text = self._get(params=params) 1070 if not text: 1071 return None 1072 1073 # The columns every cseries reply starts with, before the timestamps 1074 fixed_columns = ("parent", "child", "child description", "attribute") 1075 data: dict[str, dict[str, dict[str, Any]]] = {} 1076 unreadable = [] 1077 rows = cast(list[dict[str, str]], self._parse_csv(text, header=True)) 1078 for row in rows: 1079 parent = row.get("parent") 1080 child_name = row.get("child") 1081 if not parent or not child_name: 1082 continue 1083 # Everything after the fixed columns is a timestamped reading, in 1084 # order, because the reader keeps the header's column order 1085 readings = [ 1086 (column, value) 1087 for column, value in row.items() 1088 if column not in fixed_columns and value 1089 ] 1090 entry: dict[str, Any] = { 1091 "attribute": row.get("attribute", attribute), 1092 "value": None, 1093 "time": None, 1094 } 1095 if readings: 1096 column, value = readings[-1] 1097 try: 1098 entry["value"] = float(value) 1099 except ValueError: 1100 unreadable.append(f"{parent} {child_name} = {value}") 1101 continue 1102 try: 1103 entry["time"] = pytz.timezone(self.server_timezone).localize( 1104 datetime.strptime(column, "%Y-%m-%d %H:%M") 1105 ) 1106 except ValueError: 1107 # A column heading in a shape this does not recognize is 1108 # not worth losing the reading over 1109 entry["time"] = None 1110 data.setdefault(parent, {})[child_name] = entry 1111 1112 if unreadable: 1113 logger.warning( 1114 "Could not read {} of {} {} values from akips, those are " 1115 "missing from the result. First: {}".format( 1116 len(unreadable), 1117 len(unreadable) + len(rows), 1118 attribute, 1119 unreadable[0], 1120 ) 1121 ) 1122 if not data: 1123 return None 1124 logger.debug("Found readings for {} devices".format(len(data))) 1125 return data 1126 1127 def get_aggregate( 1128 self, 1129 period: str = "last1h", 1130 device: str = "*", 1131 attribute: str = "*", 1132 operator: str = "avg", 1133 time_interval: int = 300, 1134 group_filter: str = "any", 1135 groups: list[str] | None = None, 1136 labeled: bool = False, 1137 ) -> list[str] | list[dict[str, Any]] | None: 1138 """ 1139 Pull aggregate counter values over a period of time with optional filtering 1140 by device, attribute, and/or group membership. Defaults to all devices 1141 and attributes over the last hour with average aggregation every 300 seconds. Review 1142 AKiPS documentation for details on time filter syntax. 1143 1144 The aggregate collapses every matching device into a single series, so 1145 unlike get_series() the reply says nothing about what was measured. 1146 AKiPS sends no timestamps with it either, only the numbers, and there 1147 is one more of them than there are intervals: an hour at 300 seconds 1148 returns 13 values, not 12, the last of them landing on the end of the 1149 window. 1150 1151 Pass labeled=True to get a time against each value. That asks the 1152 server for the window with 'tf pairs' and spaces the values across it, 1153 which costs one extra request and is the only honest way to do it, 1154 since computing the axis here would be this module's clock rather than 1155 the server's. 1156 1157 An interval the server has no reading for comes back empty and is kept 1158 with a value of None, so the points stay in step with the axis. A 1159 calendar relative period returns a great many of those: 'last1d' is the 1160 whole of today, so at 300 seconds it is 288 intervals of which only the 1161 elapsed ones hold anything, and the rest are timestamped into the 1162 evening to come. Use 'last24h' for a rolling day with data throughout. 1163 1164 Supporting AKiPS command syntax: 1165 1166 aggregate [interval total|avg {secs}] time {time filter} 1167 {type} {parent regex} {child regex} {attribute regex} 1168 [profile {profile name}] [any|all|not group {group name} ...] 1169 1170 Args: 1171 period (str): time period to retrieve series from (default: 'last1h') 1172 device (str): device name or pattern to match (default: '*') 1173 attribute (str): attribute name or pattern to match (default: '*') 1174 operator (str): aggregation operator, 'avg' or 'total seconds' (default: 'avg') 1175 time_interval (int): seconds per aggregation point (default: 300). 1176 Named to match get_series() and get_latest_values(), which 1177 take the same thing 1178 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 1179 groups (list): list of group names to filter by (if any) 1180 labeled (bool): put a time against each value (default: False). 1181 Adds a request, and needs a period covering one continuous 1182 range; a filter such as 'lastweek; mon to fri 8:00 to 17:00' 1183 is several disjoint ranges and cannot be one axis 1184 Returns: 1185 A list of aggregate values, one more than the number of intervals, 1186 or None if no data found. With labeled=True, a list of 1187 dictionaries with 'time' and 'value', where 'time' is timezone 1188 aware in the server's timezone and 'value' is a float, or None 1189 where the value was not a number 1190 Raises: 1191 ValueError: if labeled is asked for and the period does not 1192 describe one continuous range 1193 AkipsError: if the AKiPS server returns an error 1194 """ 1195 params = { 1196 "cmds": f"aggregate interval {operator} {time_interval} time {period} * {device} * {attribute}" 1197 } 1198 if groups: 1199 group_list = " ".join(groups) 1200 params["cmds"] += f" {group_filter} group {group_list}" 1201 text = self._get(params=params) 1202 if text: 1203 # One CSV row of values, followed by a blank line 1204 rows = cast(list[list[str]], self._parse_csv(text)) 1205 values = rows[0] if rows else [] 1206 if not values: 1207 return None 1208 logger.debug("Found {} aggregate values".format(len(values))) 1209 if not labeled: 1210 return values 1211 return self._label_aggregate(values, period, time_interval) 1212 return None 1213 1214 def _label_aggregate( 1215 self, values: list[str], period: str, time_interval: int 1216 ) -> list[dict[str, Any]]: 1217 """ 1218 Put a time against each value of an aggregate. 1219 1220 The bounds come from the server rather than from arithmetic here, so 1221 the axis is the window AKiPS actually measured. 'tf pairs' answers 1222 with '{start},{end}' in epoch seconds, one line per continuous range 1223 within the filter. 1224 """ 1225 text = self._get(params={"cmds": f"tf pairs {period}"}) 1226 ranges = self._parse_lines(text or "") 1227 if len(ranges) != 1: 1228 raise ValueError( 1229 "Cannot label aggregate values for period {!r}: it describes " 1230 "{} separate ranges, and a single time axis needs one".format( 1231 period, len(ranges) 1232 ) 1233 ) 1234 try: 1235 start_epoch, _end_epoch = (int(field) for field in ranges[0].split(",")) 1236 except ValueError: 1237 raise ValueError( 1238 "Could not read the window for period {!r} from {!r}; expected " 1239 "'{{start}},{{end}}' in epoch seconds".format(period, ranges[0]) 1240 ) from None 1241 1242 timezone = pytz.timezone(self.server_timezone) 1243 labeled = [] 1244 unreadable = [] 1245 empty = 0 1246 for index, value in enumerate(values): 1247 # Kept rather than dropped either way: leaving a point out would 1248 # shift every one after it along the axis 1249 reading: float | None = None 1250 if not value.strip(): 1251 # An interval the server has no data for. A calendar relative 1252 # period such as 'last1d' covers the whole day, so every bucket 1253 # after the current moment comes back empty; that is the period 1254 # doing what it says rather than anything wrong with the reply 1255 empty += 1 1256 else: 1257 try: 1258 reading = float(value) 1259 except ValueError: 1260 unreadable.append(value) 1261 labeled.append( 1262 { 1263 "time": datetime.fromtimestamp( 1264 start_epoch + index * time_interval, tz=timezone 1265 ), 1266 "value": reading, 1267 } 1268 ) 1269 if empty: 1270 logger.debug( 1271 "{} of {} aggregate intervals had no data, kept with a value " 1272 "of None".format(empty, len(values)) 1273 ) 1274 if unreadable: 1275 logger.warning( 1276 "Could not read {} of {} aggregate values as numbers, those " 1277 "points are kept with a value of None. First: {!r}".format( 1278 len(unreadable), len(values), unreadable[0][:100] 1279 ) 1280 ) 1281 return labeled 1282 1283 # Low-level operations, kept for compatibility 1284 1285 def cmd(self, cmd: str, output: str = "raw") -> str | None: 1286 """ 1287 Deprecated since 1.0.0, use call() instead, which reaches every API 1288 section and can parse the reply rather than only returning it raw. 1289 1290 Args: 1291 cmd (str): AKiPS command string to send 1292 output (str): desired output format, only 'raw' is supported 1293 Returns: 1294 The command output, or None if no output 1295 Raises: 1296 ValueError: if an invalid output format is provided 1297 AkipsError: if the AKiPS server returns an error 1298 """ 1299 warnings.warn( 1300 "cmd() is deprecated and will be removed in a future release, " 1301 "use call() instead", 1302 DeprecationWarning, 1303 stacklevel=2, 1304 ) 1305 if output != "raw": 1306 raise ValueError("Invalid output value provided to cmd.") 1307 return cast("str | None", self.call(command=cmd)) 1308 1309 # --------------------------------------------------------------------------- 1310 # api-script methods, these require the 'api-rw' user 1311 1312 def get_device_by_ip(self, ipaddr: str) -> str | None: 1313 """ 1314 Return the device name (primary key) for a device matching the given IP address. 1315 AKiPS records additional IP addresses when found on devices, so this function 1316 can be used to find the primary device name (primary key) from any known IP address. 1317 1318 A device is stored under one address, but it may answer on several. 1319 The case this exists for is a syslog message or SNMP trap arriving 1320 from an interface other than the one AKiPS knows the device by, where 1321 the source address matches no device at all. AKiPS keeps an internal 1322 address to device table, which the site script reads, so this maps any 1323 address the server has seen back to the device holding it. 1324 1325 That table is not the device's attributes, and searching the attributes 1326 is not a substitute. AKiPS keeps the addresses configured on a device 1327 in a CSV file rather than in its database, which is why an mget against 1328 attributes does not find them; the GUI shows the same file as its 1329 'Device to IP Mapping' report. Confirmed on a live server, where an 1330 address resolved here to a device whose entire attribute tree contained 1331 no mention of it. 1332 1333 This is the one read in this module that a read only deployment cannot 1334 perform. AKiPS exposes it as a site script rather than a database 1335 query, so it lives in api-script and needs rw_password even though it 1336 changes nothing. A client holding only ro_password raises 1337 AkipsCredentialError rather than returning None, so a caller building 1338 on it should not plan for a read only deployment. 1339 1340 **This call is sent as GET, not POST**, so its password travels in 1341 the query string. api-script does not answer a POST: the server sends 1342 no body and holds the connection open until the client gives up. See 1343 SECTION_METHODS, which is where to say so if a server of yours does 1344 take a POST on this section. 1345 1346 Supporting AKiPS site script function (which requires the api-rw user): 1347 1348 web_find_device_by_ip(ipaddr) 1349 1350 Args: 1351 ipaddr (str): IP address to search for 1352 Returns: 1353 the device name (str) if found, or None if no match is found 1354 Raises: 1355 AkipsError: if the AKiPS server returns an error 1356 """ 1357 params = {"function": "web_find_device_by_ip", "ipaddr": ipaddr} 1358 text = self._get(section="api-script", params=params) 1359 if not text: 1360 return None 1361 for line in text.split("\n"): 1362 match = re.match(r"IP Address (\S+) is configured on (\S+)", line) 1363 if match: 1364 address = match.group(1) 1365 device_name = match.group(2) 1366 logger.debug(f"Found {address} on device {device_name}") 1367 return device_name 1368 # The site script says so in as many words when it finds nothing, so a 1369 # reply that is neither that nor a match did not come from it. The 1370 # likeliest cause is the script not being installed, which would 1371 # otherwise read as 'no device has that address' and be believed. 1372 if "is not configured on any devices" not in text: 1373 logger.warning( 1374 "web_find_device_by_ip returned something unexpected; check " 1375 "that the site script is installed on this AKiPS server. " 1376 "Reply: {}".format(self._redact_text(text.strip()[:200])) 1377 ) 1378 return None 1379 1380 def set_group_membership(self, device: str, group: str, mode: str) -> None: 1381 """ 1382 Update manual grouping rules for a device, including the special 'maintenance_mode' 1383 group. The web api script fails silently if the device or group does not exist. 1384 1385 **This call is sent as GET, not POST**, so its password travels in 1386 the query string. api-script does not answer a POST: the server sends 1387 no body and holds the connection open until the client gives up. See 1388 SECTION_METHODS, which is where to say so if a server of yours does 1389 take a POST on this section. 1390 1391 Supporting AKiPS site script function (which requires the api-rw user): 1392 1393 web_manual_grouping(type, group, mode, device) 1394 1395 Args: 1396 device (str): the AKiPS name of one device, exactly; this 1397 takes no pattern 1398 group (str): group name to update 1399 mode (str): 'assign' to add device to group, 'clear' to remove device from group 1400 Returns: 1401 None 1402 Raises: 1403 ValueError: if invalid parameters are provided 1404 AkipsError: if the AKiPS server returns an error 1405 """ 1406 if not device: 1407 raise ValueError( 1408 "a valid device name must be provided for manual grouping update" 1409 ) 1410 if not group: 1411 raise ValueError( 1412 "a valid group name must be provided for manual grouping update" 1413 ) 1414 if mode not in ("assign", "clear"): 1415 raise ValueError( 1416 "mode must be 'assign' or 'clear' for manual grouping update" 1417 ) 1418 params = { 1419 "function": "web_manual_grouping", 1420 "type": "device", 1421 "group": group, # group_name 1422 "mode": mode, # 'assign' or 'clear' for device memberships 1423 "device": device, # device_name 1424 } 1425 text = self._get(section="api-script", params=params) 1426 if text: 1427 logger.error("Web API request failed: {}".format(text)) 1428 raise AkipsError(message=text) 1429 return None 1430 1431 SCRIPT_TIMEOUT = 300 1432 """Seconds a site script that does work is given, in place of the 1433 client's timeout. 1434 1435 Site scripts divide into two kinds. Most answer a question and return at 1436 once — get_device_by_ip() and set_group_membership() are ordinary 1437 requests and keep the client's timeout, so a hung one fails as promptly 1438 as any other call. A few go away and do something: deleting a device 1439 today, and discovery, rewalk and rename if those are ever wrapped. Those 1440 are what this is for. 1441 1442 It is not per method on purpose. Every long running script wants the 1443 same thing — more room than a read gets — and a constant for each would 1444 be a new name to learn for every script added. A method needing 1445 something different takes a timeout argument instead. 1446 1447 Why generous: a timeout part way through work that changes the server 1448 leaves the worst of the three outcomes, where the caller cannot tell 1449 whether it happened, since nothing can confirm an outcome when the call 1450 itself raises. Waiting longer costs only waiting. 1451 1452 How long any of these really take is not known. The one timing on 1453 record, a delete just past 30 seconds against a 30 second timeout, was 1454 taken while api-script still hung on every POST, so it measures the 1455 client giving up rather than the work — see SECTION_METHODS. 1456 1457 A client configured with a longer timeout than this keeps it; this is a 1458 floor, not a ceiling.""" 1459 1460 def delete_device(self, device: str, timeout: int | None = None) -> bool: 1461 """ 1462 Delete one device from AKiPS. 1463 1464 **This cannot be undone.** Whether the samples, events and 1465 availability held against the device go with it is a property of 1466 AKiPS's own config_delete_device built in, which the site script calls 1467 and this module cannot see into, so treat the whole record as lost 1468 until AKiPS says otherwise. There is no merge: where the same box is 1469 registered twice under two names, copy whatever the surviving record 1470 should keep before deleting the other one, because nothing moves 1471 across on its own. 1472 1473 **This call is sent as GET, not POST**, so its password travels in 1474 the query string. api-script does not answer a POST: the server sends 1475 no body and holds the connection open until the client gives up. See 1476 SECTION_METHODS, which is where to say so if a server of yours does 1477 take a POST on this section. 1478 1479 Supporting AKiPS site script function (which requires the api-rw user): 1480 1481 web_delete_device(device_names) 1482 1483 AKiPS publishes that script and does not install it by default; see 1484 akips_setup/README.md. It prints nothing whether it worked or not, so 1485 this method confirms the outcome rather than trusting the silence. It 1486 checks the device is there first, which is how a name that never 1487 existed is told apart from one that was removed, and checks it is gone 1488 afterwards, which is how a script that quietly did nothing is caught. 1489 That costs two extra requests, which is the right trade for an 1490 operation with no undo. 1491 1492 **An exception does not mean nothing happened.** The confirmation 1493 below cannot run when the call itself fails, and AKiPS finishes the 1494 work whether or not the client is still listening: a delete that ran 1495 just past a 30 second timeout removed the device and raised anyway, 1496 so the caller recorded a failure against a device already gone. On any exception, ask AKiPS again rather than 1497 recording a failure. Gone, still there, and could not tell are three 1498 different outcomes and only the first two are knowable from here. 1499 1500 Args: 1501 device (str): the AKiPS name of one device, exactly. This takes 1502 no pattern and no list. A name holding a comma or an asterisk 1503 is refused: the site script splits its argument on commas, so 1504 such a name would delete more than was asked for, and a 1505 partial or oversized delete cannot be walked back. 1506 timeout (int): seconds to wait for the delete itself. Defaults 1507 to SCRIPT_TIMEOUT, or the client's timeout if that is longer, 1508 because a timeout during a destructive call leaves an outcome 1509 nobody can read. The two lookups either side are ordinary 1510 reads and use the client's timeout. 1511 Returns: 1512 True if the device was deleted, False if there was no such device. 1513 The two are distinguishable on purpose, so a caller does not 1514 report success for a name that was never there. 1515 Raises: 1516 ValueError: if device is empty, is a pattern, or could name more 1517 than one device 1518 AkipsCredentialError: if no rw_password was given to AKIPS() 1519 AkipsError: if AKiPS returns an error, or if the device is still 1520 present afterwards 1521 """ 1522 if not device: 1523 raise ValueError("a device name must be provided to delete a device") 1524 if device.startswith("/") and device.endswith("/") and len(device) > 1: 1525 raise ValueError( 1526 "delete_device takes one device name, not a pattern. Got " 1527 "{!r}".format(device) 1528 ) 1529 for char in (",", "*"): 1530 if char in device: 1531 # web_delete_device does cgi_param("device_names") in scalar 1532 # context and splits on commas itself, so a comma here is not 1533 # an odd name but a second device. Refused rather than 1534 # escaped, because there is no undo to fall back on. 1535 raise ValueError( 1536 "refusing to delete {!r}: a name containing {!r} can match " 1537 "more than one device, and this cannot be undone".format( 1538 device, char 1539 ) 1540 ) 1541 1542 # Checked before anything is looked up, so a client with no rw 1543 # password fails on the credential rather than after spending a 1544 # request on a delete it could never have made. 1545 self._credentials_for("api-script") 1546 1547 if self.get_device(device) is None: 1548 logger.info("No AKiPS device named {!r}, nothing to delete".format(device)) 1549 return False 1550 1551 params = { 1552 "function": "web_delete_device", 1553 "device_names": device, # one name; the script splits on commas 1554 } 1555 if timeout is None: 1556 # A floor rather than a replacement: a client deliberately given 1557 # longer than this keeps it. 1558 timeout = max(self.timeout, self.SCRIPT_TIMEOUT) 1559 text = self._get(section="api-script", params=params, timeout=timeout) 1560 if text: 1561 logger.error("Web API request failed: {}".format(text)) 1562 raise AkipsError(message=text) 1563 1564 if self.get_device(device) is not None: 1565 raise AkipsError( 1566 message=( 1567 "AKiPS still holds a device named {!r} after " 1568 "web_delete_device returned nothing. Check that the site " 1569 "script is installed and that api-rw is allowed to run " 1570 "it".format(device) 1571 ) 1572 ) 1573 logger.info("Deleted AKiPS device {!r} and its history".format(device)) 1574 return True 1575 1576 # --------------------------------------------------------------------------- 1577 # api-msg methods, these require the 'api-ro' user 1578 1579 # The message types AKiPS keeps, and what get_msg accepts. None asks for 1580 # both, which is what the api-msg section returns when the parameter is 1581 # left off. 1582 MSG_TYPES = ("syslog", "trap") 1583 """The message types get_msg() accepts. None asks for both.""" 1584 1585 # 'period' and 'msg_type' map to the AKiPS query parameters 'time' and 1586 # 'type'. They are deliberately named apart from those, because 'type' is 1587 # a builtin and 'time' a standard library module, and because 'period' is 1588 # what the rest of this module already calls a time filter. 1589 def get_msg( 1590 self, 1591 period: str = "last1h", 1592 addr: str | None = None, 1593 msg_type: str | None = None, 1594 device: str | None = None, 1595 regex: str | None = None, 1596 limit: int | None = None, 1597 ) -> list[dict[str, str]] | None: 1598 """ 1599 Retrieve syslog or trap messages from the AKiPS api-msg database. The api-msg 1600 access requires username to be 'api-ro'. 1601 1602 Supporting AKiPS web API syntax: 1603 1604 https://{server}/api-msg?password={pw};time={time filter}; 1605 [addr={ip filter}];[type=syslog|trap];[device={name}|{regex}]; 1606 [regex={regex filter}];[limit={qty messages}] 1607 1608 This is the highest volume call here, and worth filtering. Measured on 1609 a 17,000 device fleet, an unfiltered 'last1h' returned 472,014 messages 1610 in 5.5 seconds; the same hour asking only for traps returned 5,160 in 1611 0.8 seconds. Syslog is the bulk of it, and a single appliance can be a 1612 large share of that on its own. See get_traps() and get_syslog(). 1613 1614 Args: 1615 period (str): Required, time period to retrieve messages from 1616 (default: 'last1h'). 'lastNm' and 'lastNh' are rolling 1617 windows of the length they name, measured back from the moment 1618 of the call. 'lastNd' is calendar relative, meaning N-1 whole 1619 days plus today so far, so 'last1d' is today rather than 24 1620 hours; use 'last24h' for a rolling day 1621 addr (str): IP address to filter messages by (default: None) 1622 msg_type (str): message type, 'syslog' or 'trap', or None for both 1623 (default: None). See get_syslog() and get_traps(), which name 1624 the type rather than asking a caller to spell it 1625 device (str): device name to filter messages by (default: None) 1626 regex (str): regex pattern to filter message content by (default: None) 1627 limit (int): maximum number of messages to return (default: None). 1628 AKiPS fills this from the start of the window, so by default it 1629 returns the oldest matching messages rather than the newest, 1630 and there is no ordering parameter on the request. AKiPS 25.6 1631 added a reverse sort option under Miscellaneous Settings, which 1632 is server wide rather than per call; whether it reaches this 1633 section has not been tested here. For recent activity narrow 1634 'period' instead: 'last15m' with no limit costs far less than 1635 an hour of messages thrown away after the fact 1636 Returns: 1637 A list of dictionaries, each with 'time', 'type', 'ip_ver', 1638 'ip_addr' and 'message', or None if no data found. 'ip_addr' is 1639 where the message came from, which need not be the address AKiPS 1640 holds for the device: a device with several interfaces can send 1641 from any of them. get_device_by_ip() resolves one to a device 1642 Raises: 1643 ValueError: if msg_type is not 'syslog', 'trap' or None 1644 AkipsError: if the AKiPS server returns an error 1645 """ 1646 1647 # Checked rather than quietly ignored. An unrecognized type used to be 1648 # dropped, so a caller asking for 'traps' or 'Syslog' was sent no type 1649 # at all and got both back believing it had filtered to one. 1650 if msg_type is not None and msg_type not in self.MSG_TYPES: 1651 raise ValueError( 1652 "Invalid msg_type provided to get_msg, expected one of {}, " 1653 "or None for both".format(", ".join(self.MSG_TYPES)) 1654 ) 1655 params = {"time": period} 1656 if msg_type is not None: 1657 params["type"] = msg_type 1658 if addr: 1659 params["addr"] = addr 1660 if device: 1661 params["device"] = device 1662 if regex: 1663 params["regex"] = regex 1664 if limit: 1665 params["limit"] = str(limit) 1666 text = self._get(section="api-msg", params=params) 1667 if text: 1668 # Each syslog or trap message contains: 1669 # header line: {system timestamp} {type} {IP version} {IP address} 1670 # message line(s): {message text} 1671 # blank terminating line 1672 # 1673 # Records are split on that blank line rather than by recognizing 1674 # each header, because a body line can look exactly like a header 1675 # and would otherwise start a new record in the middle of a 1676 # message, turning one message into two with empty bodies. 1677 data = [] 1678 unparsed = 0 1679 for record in re.split(r"\n\s*\n", text): 1680 lines = [line for line in record.split("\n") if line.strip()] 1681 if not lines: 1682 continue 1683 header = re.match( 1684 r"^(?P<time>\S+)\s(?P<type>\S+)\s(?P<ip_ver>[46])\s(?P<ip_addr>\S+)$", 1685 lines[0], 1686 ) 1687 if not header: 1688 unparsed += 1 1689 continue 1690 data.append( 1691 { 1692 "time": header.group("time"), 1693 "type": header.group("type"), 1694 "ip_ver": header.group("ip_ver"), 1695 "ip_addr": header.group("ip_addr"), 1696 # Everything after the header is the message, whatever 1697 # any of those lines happen to look like 1698 "message": "\n".join(lines[1:]), 1699 } 1700 ) 1701 if unparsed: 1702 logger.warning( 1703 "Could not parse {} of {} message records from akips, " 1704 "those messages are missing from the result".format( 1705 unparsed, unparsed + len(data) 1706 ) 1707 ) 1708 # A reply that parses to nothing is nothing found, the same 1709 # answer an empty reply gives, rather than an empty container 1710 if not data: 1711 return None 1712 logger.debug("Found {} messages in akips".format(len(data))) 1713 return data 1714 return None 1715 1716 def get_syslog( 1717 self, 1718 period: str = "last1h", 1719 addr: str | None = None, 1720 device: str | None = None, 1721 regex: str | None = None, 1722 limit: int | None = None, 1723 ) -> list[dict[str, str]] | None: 1724 """ 1725 Retrieve syslog messages, leaving traps out. 1726 1727 The same as get_msg(msg_type='syslog') with every other filter 1728 forwarded, named so the type does not have to be spelled correctly to 1729 take effect. 1730 1731 Syslog is the high volume half of api-msg: an unfiltered hour was 1732 465,936 messages on a 17,000 device fleet, one appliance accounting 1733 for a large share of it. Pass a shorter period, a device or a regex 1734 unless the whole of it is wanted. 1735 1736 Args: 1737 period (str): time period to retrieve messages from 1738 (default: 'last1h'). 'lastNm' and 'lastNh' are rolling 1739 windows of the length they name; 'lastNd' is calendar 1740 relative, so 'last1d' is today rather than 24 hours 1741 addr (str): IP address to filter messages by (default: None) 1742 device (str): device name to filter messages by (default: None) 1743 regex (str): regex pattern to filter message content by 1744 (default: None) 1745 limit (int): maximum number of messages to return (default: None). 1746 This returns the oldest matching messages by default, not the 1747 newest; narrow 'period' for recent activity. See get_msg() 1748 for the server setting that may reverse it 1749 Returns: 1750 A list of dictionaries, each with 'time', 'type', 'ip_ver', 1751 'ip_addr' and 'message', or None if no data found. 'ip_addr' is 1752 where the message came from, which need not be the address AKiPS 1753 holds for the device: a device with several interfaces can send 1754 from any of them. get_device_by_ip() resolves one to a device 1755 Raises: 1756 AkipsError: if the AKiPS server returns an error 1757 """ 1758 return self.get_msg( 1759 period=period, 1760 msg_type="syslog", 1761 addr=addr, 1762 device=device, 1763 regex=regex, 1764 limit=limit, 1765 ) 1766 1767 def get_traps( 1768 self, 1769 period: str = "last1h", 1770 addr: str | None = None, 1771 device: str | None = None, 1772 regex: str | None = None, 1773 limit: int | None = None, 1774 ) -> list[dict[str, str]] | None: 1775 """ 1776 Retrieve SNMP traps, leaving syslog out. 1777 1778 The same as get_msg(msg_type='trap') with every other filter 1779 forwarded, named so the type does not have to be spelled correctly to 1780 take effect. 1781 1782 Asking for traps is what makes this call cheap enough to poll: on a 1783 17,000 device fleet an hour of traps was 5,160 messages against 1784 472,014 for an unfiltered hour. 1785 1786 The body of a trap is a varbind list, one per line, as 1787 '{module} {attribute} {instance} {type} {value}'. It is returned as 1788 the raw 'message' text; this does not split it up. 1789 1790 Args: 1791 period (str): time period to retrieve messages from 1792 (default: 'last1h'). 'lastNm' and 'lastNh' are rolling 1793 windows of the length they name; 'lastNd' is calendar 1794 relative, so 'last1d' is today rather than 24 hours 1795 addr (str): IP address to filter messages by (default: None) 1796 device (str): device name to filter messages by (default: None) 1797 regex (str): regex pattern to filter message content by 1798 (default: None) 1799 limit (int): maximum number of messages to return (default: None). 1800 This returns the oldest matching messages by default, not the 1801 newest; narrow 'period' for recent activity. See get_msg() 1802 for the server setting that may reverse it 1803 Returns: 1804 A list of dictionaries, each with 'time', 'type', 'ip_ver', 1805 'ip_addr' and 'message', or None if no data found. 'ip_addr' is 1806 where the message came from, which need not be the address AKiPS 1807 holds for the device: a device with several interfaces can send 1808 from any of them. get_device_by_ip() resolves one to a device 1809 Raises: 1810 AkipsError: if the AKiPS server returns an error 1811 """ 1812 return self.get_msg( 1813 period=period, 1814 msg_type="trap", 1815 addr=addr, 1816 device=device, 1817 regex=regex, 1818 limit=limit, 1819 ) 1820 1821 # --------------------------------------------------------------------------- 1822 # api-availability methods, these require the 'api-ro' user 1823 1824 # AKiPS has two kinds of time filter and they are easy to confuse. 1825 # 'lastNd' is calendar relative: it means N-1 whole days plus today so 1826 # far, so 'last1d' is today and measures minutes just after midnight. 1827 # 'lastNh' and 'lastNm' are rolling windows of the length they name. 1828 # 1829 # These methods default to the rolling form. An availability figure is a 1830 # percentage of the window it was measured over, and a caller asking for 1831 # 'the last day' and rendering the answer should not silently get a 1832 # five minute sample that reads as a reliable 100% every night. 1833 AVAILABILITY_PERIOD = "last24h" 1834 """The period the availability methods use by default, a rolling 24 1835 hours rather than 'last1d', which AKiPS reads as today so far.""" 1836 1837 def get_group_availability( 1838 self, 1839 period: str = AVAILABILITY_PERIOD, 1840 report: str = "ping4", 1841 group: str | None = None, 1842 ) -> list[dict[str, str]] | None: 1843 """ 1844 Retrieve availability statistics for a group of devices over a time period. 1845 1846 # output format: {child},{attr},{group name},{total time},{match time},{group target},{tf}[;{group tf}] 1847 # example: nm-availability mode group time last1w report ping4 1848 1849 ping4,PING.icmpState,1-Building-4,11688115,11687711,9990,last1w 1850 ping4,PING.icmpState,1-Fraser,8213270,8213190,9990,last1w 1851 ping4,PING.icmpState,1-Building-16,44541195,44540002,9990,last1w 1852 ping4,PING.icmpState,Accedian,1766635,1766635,9890,last1w;mon to sat 6:00 to 20:00 1853 ping4,PING.icmpState,Aerohive,589475,589475,9999,last1w;mon to fri 7:00 to 19:00; sat 8:00 to 18:00 1854 1855 Args: 1856 period (str): time filter, refer to the AKiPS programming guide 1857 (default: 'last24h'). 'lastNd' is calendar relative, meaning 1858 N-1 whole days plus today so far, so 'last1d' is today rather 1859 than 24 hours and shrinks to minutes just after midnight. 1860 'lastNh' and 'lastNm' are rolling windows of the length they 1861 name. 'total time' in the reply is the window measured 1862 report (str): 'ping4', 'ping6', 'snmp' or 'ifstatus', in any 1863 combination, comma separated (default: 'ping4') 1864 group (str): group name to filter by, or every group 1865 Returns: 1866 A list of dictionaries, one per group, or None if nothing matched 1867 Raises: 1868 AkipsError: if the AKiPS server returns an error 1869 """ 1870 params = { 1871 "maintenance": "off", # 'on' or 'off', show/hide maintenance mode devices 1872 "mode": "group", # 'group', 'device' or 'events' 1873 "time": period, # time filter, refer to programming guide 1874 "report": report, # 'ping4', 'ping6', 'snmp', 'ifstatus'. Any combination, comma separated, 1875 # "entity": device, # {device} [{child}] to filter by device or child 1876 "group": group, # {group name} to filter by group 1877 # "profile": "" # {profile name} to filter by profile 1878 } 1879 text = self._get(section="api-availability", params=params) 1880 if text: 1881 # This endpoint sends no header row, so the column names come from 1882 # here rather than from the reply 1883 column_headers = [ 1884 "child", 1885 "attr", 1886 "group name", 1887 "total time", 1888 "match time", 1889 "group target", 1890 "tf", 1891 ] 1892 csv_to_list = self._parse_csv(text, fieldnames=column_headers) 1893 logger.debug("Found {} entries".format(len(csv_to_list))) 1894 return cast("list[dict[str, str]]", csv_to_list) 1895 return None 1896 1897 def get_device_availability( 1898 self, 1899 period: str = AVAILABILITY_PERIOD, 1900 report: str = "ping4", 1901 device: str | None = None, 1902 group: str | None = None, 1903 ) -> list[dict[str, str]] | None: 1904 """ 1905 Retrieve availability statistics per device over a time period. 1906 1907 Where group mode summarises a whole group, this reports each device 1908 and child separately, so a device checked by both ping and SNMP 1909 appears on two rows. 1910 1911 A device or a group is required. Unlike group mode, device mode 1912 answers an unscoped call with an empty body rather than an error, 1913 which would reach the caller as None and read as 'nothing to report'. 1914 1915 # output format: {parent},{child},{attr},{total time},{match time},{group target} 1916 # example: nm-availability mode device time last1w report snmp,ping4 group Accedian 1917 1918 accedian-131-2-7,ping4,PING.icmpState,136020,136020,9890 1919 accedian-131-2-7,sys,SNMP.snmpState,136020,136020,9890 1920 accedian-131-2-8,ping4,PING.icmpState,136020,136020,9890 1921 accedian-131-2-8,sys,SNMP.snmpState,136020,136020,9890 1922 1923 'group target' is the availability AKiPS is configured to expect, in 1924 basis points, so 9890 is 98.90% and 10000 is 100.00%. It is set per 1925 group, so a caller can report against the target already agreed on 1926 the server rather than inventing a threshold of its own. 1927 1928 Args: 1929 period (str): time filter, refer to the AKiPS programming guide 1930 (default: 'last24h'). 'lastNd' is calendar relative, meaning 1931 N-1 whole days plus today so far, so 'last1d' is today rather 1932 than 24 hours and shrinks to minutes just after midnight. 1933 'lastNh' and 'lastNm' are rolling windows of the length they 1934 name. 'total time' in the reply is the window measured 1935 report (str): 'ping4', 'ping6', 'snmp' or 'ifstatus', in any 1936 combination, comma separated (default: 'ping4') 1937 device (str): device to filter by, as '{device}' or 1938 '{device} {child}'. This is the device's AKiPS name 1939 exactly, its one primary key, which is either its sysName or 1940 its IP address depending on how the server names devices, and 1941 takes no pattern; get_device_by_ip() resolves an address to it 1942 group (str): group name to filter by 1943 Returns: 1944 A list of dictionaries, one per device and child, or None if 1945 nothing matched 1946 Raises: 1947 ValueError: if neither device nor group is given 1948 AkipsError: if the AKiPS server returns an error 1949 """ 1950 # Checked before the request, so a call that could only ever come back 1951 # empty fails as the mistake it is rather than as good news 1952 if device is None and group is None: 1953 raise ValueError( 1954 "get_device_availability needs a device or a group to scope it, " 1955 "an unscoped call returns nothing at all" 1956 ) 1957 params = { 1958 "maintenance": "off", # 'on' or 'off', show/hide maintenance mode devices 1959 "mode": "device", # 'group', 'device' or 'events' 1960 "time": period, # time filter, refer to programming guide 1961 "report": report, # 'ping4', 'ping6', 'snmp', 'ifstatus'. Any combination, comma separated, 1962 # 'entity' is not in the nm-availability syntax the AKiPS API 1963 # guide publishes, which lists only mode, time, report, group and 1964 # profile. It works, and is how device and event mode are scoped 1965 # here, but being undocumented it is the parameter most likely to 1966 # change under us in a future AKiPS release. 1967 "entity": device, # {device} [{child}] to filter by device or child 1968 "group": group, # {group name} to filter by group 1969 } 1970 text = self._get(section="api-availability", params=params) 1971 if text: 1972 # This endpoint sends no header row, so the column names come from 1973 # here rather than from the reply. They are not group mode's 1974 # columns; each mode of nm-availability returns its own. 1975 column_headers = [ 1976 "parent", 1977 "child", 1978 "attr", 1979 "total time", 1980 "match time", 1981 "group target", 1982 ] 1983 csv_to_list = self._parse_csv(text, fieldnames=column_headers) 1984 logger.debug("Found {} entries".format(len(csv_to_list))) 1985 return cast("list[dict[str, str]]", csv_to_list) 1986 return None 1987 1988 def get_event_availability( 1989 self, 1990 period: str = AVAILABILITY_PERIOD, 1991 report: str = "ping4", 1992 device: str | None = None, 1993 group: str | None = None, 1994 ) -> list[dict[str, str]] | None: 1995 """ 1996 Retrieve the up and down event pairs behind a device's availability. 1997 1998 Where device mode gives the totals, this gives the outages that 1999 produced them, one row per pair. 2000 2001 # output format: {parent},{child},{down},{up},{total time},{match time} 2002 # example: nm-availability mode events time last1M report ping4 entity cisco-131-16-1 2003 2004 cisco-131-16-1,ping4,1603822871,1603822916,2389764,2388341 2005 cisco-131-16-1,ping4,1603088563,1603089823,2389764,2388341 2006 cisco-131-16-1,ping4,1603060380,1603060498,2389764,2388341 2007 2008 'down' and 'up' are epoch seconds bounding a single outage, so a 2009 device that went down twice comes back as two rows. Both are empty 2010 for a device that stayed up, which still reports the window it was 2011 measured over. 2012 2013 Take the length of an outage as 'up' minus 'down'. 'total time' and 2014 'match time' describe the measurement rather than the row they sit 2015 beside: every row in a reply carries the same 'total time', the 2016 length of the window, and a device's 'match time' is that less the 2017 time it spent down. Measured against a live server, a device with 2018 outages of 44 and 46 seconds came back with a 'match time' 90 below 2019 'total time' on both of its rows, while devices in the same reply 2020 that stayed up had the two equal. 2021 2022 So 'total time' is not the length of the outage on its row. Reading 2023 it that way gives the whole measurement window as the duration of a 2024 one minute flap. 2025 2026 These columns are not the ones group or device mode returns, so the 2027 three modes are parsed separately rather than sharing a field list. 2028 2029 Args: 2030 period (str): time filter, refer to the AKiPS programming guide 2031 (default: 'last24h'). 'lastNd' is calendar relative, meaning 2032 N-1 whole days plus today so far, so 'last1d' is today rather 2033 than 24 hours and shrinks to minutes just after midnight. 2034 'lastNh' and 'lastNm' are rolling windows of the length they 2035 name. 'total time' in the reply is the window measured 2036 report (str): 'ping4', 'ping6', 'snmp' or 'ifstatus', in any 2037 combination, comma separated (default: 'ping4') 2038 device (str): device to filter by, as '{device}' or 2039 '{device} {child}'. This is the device's AKiPS name 2040 exactly, its one primary key, which is either its sysName or 2041 its IP address depending on how the server names devices, and 2042 takes no pattern; get_device_by_ip() resolves an address to it 2043 group (str): group name to filter by 2044 Returns: 2045 A list of dictionaries, one per up and down pair, or None if 2046 nothing matched 2047 Raises: 2048 ValueError: if neither device nor group is given 2049 AkipsError: if the AKiPS server returns an error 2050 """ 2051 # Same as device mode, confirmed against a server: without a scope the 2052 # reply is empty rather than an error, which would arrive as None and 2053 # read as 'no outages' 2054 if device is None and group is None: 2055 raise ValueError( 2056 "get_event_availability needs a device or a group to scope it, " 2057 "an unscoped call returns nothing at all" 2058 ) 2059 params = { 2060 "maintenance": "off", # 'on' or 'off', show/hide maintenance mode devices 2061 "mode": "events", # 'group', 'device' or 'events' 2062 "time": period, # time filter, refer to programming guide 2063 "report": report, # 'ping4', 'ping6', 'snmp', 'ifstatus'. Any combination, comma separated, 2064 # 'entity' is not in the nm-availability syntax the AKiPS API 2065 # guide publishes, which lists only mode, time, report, group and 2066 # profile. It works, and is how device and event mode are scoped 2067 # here, but being undocumented it is the parameter most likely to 2068 # change under us in a future AKiPS release. 2069 "entity": device, # {device} [{child}] to filter by device or child 2070 "group": group, # {group name} to filter by group 2071 } 2072 text = self._get(section="api-availability", params=params) 2073 if text: 2074 # This endpoint sends no header row, so the column names come from 2075 # here rather than from the reply 2076 column_headers = [ 2077 "parent", 2078 "child", 2079 "down", 2080 "up", 2081 "total time", 2082 "match time", 2083 ] 2084 csv_to_list = self._parse_csv(text, fieldnames=column_headers) 2085 logger.debug("Found {} entries".format(len(csv_to_list))) 2086 return cast("list[dict[str, str]]", csv_to_list) 2087 return None 2088 2089 # --------------------------------------------------------------------------- 2090 # Generic operations, these reach any API section 2091 # 2092 # call() is not fixed to one section the way the methods above are. It 2093 # takes the section as an argument and picks the account from 2094 # SECTION_USERS, which is how the sections with no methods of their own 2095 # here are reached. 2096 2097 # The reply shapes call() can parse, mapped to the parser for each 2098 OUTPUT_FORMATS = ("raw", "lines", "key_value", "attributes", "csv", "csv_dict") 2099 """The reply shapes call() can parse.""" 2100 2101 def call( 2102 self, 2103 command: str | None = None, 2104 section: str = "api-db", 2105 params: dict[str, Any] | None = None, 2106 output: str = "raw", 2107 user: str | None = None, 2108 ) -> Any: 2109 """ 2110 Send an arbitrary request to any AKiPS web API section and parse the 2111 reply in one of the shapes AKiPS replies in. 2112 2113 This is the general purpose call for anything the specific methods do 2114 not cover. It parses with the same routines they use, so an ad-hoc 2115 query returns the same shape its dedicated method would. 2116 2117 Sections do not share a parameter vocabulary. api-db takes a command 2118 string, while api-script, api-msg and api-availability each take their 2119 own named parameters, so pass 'command' for the first and 'params' for the 2120 others. Passing both adds the command to the given parameters. 2121 2122 Output formats, and where each one occurs: 2123 2124 raw the reply unchanged, as a string 2125 lines a list of non-blank lines 2126 key_value '{key} = {value}' lines, as from mgroup 2127 attributes '{parent} {child} {attribute} = {value}' lines, as from 2128 mget, nested by parent, child, then attribute 2129 csv CSV rows as lists, for replies with no header row 2130 csv_dict CSV rows as dictionaries keyed by the header row 2131 2132 Args: 2133 command (str): command string for the api-db section, shorthand 2134 for params={'cmds': command} 2135 section (str): API section to call (default: 'api-db') 2136 params (dict): parameters for sections that take no command string 2137 output (str): one of the formats listed above (default: 'raw') 2138 user (str): force the 'ro' or 'rw' account, for a section 2139 whose requirement is not in SECTION_USERS, or a command 2140 needing more rights than its section usually does 2141 Returns: 2142 The reply in the requested shape, or None if the server returned 2143 nothing 2144 Raises: 2145 ValueError: if output is not a supported format, or if neither 2146 command nor params was provided 2147 AkipsError: if the AKiPS server returns an error 2148 """ 2149 # Check before making the request, so a bad argument fails the same way 2150 # whether or not the server returned anything 2151 if output not in self.OUTPUT_FORMATS: 2152 raise ValueError( 2153 "Invalid output value provided to call, expected one of {}".format( 2154 ", ".join(self.OUTPUT_FORMATS) 2155 ) 2156 ) 2157 if command is None and params is None: 2158 raise ValueError("call requires either a command or a params dictionary") 2159 2160 request_params = dict(params or {}) 2161 if command is not None: 2162 request_params["cmds"] = command 2163 2164 text = self._get(section=section, params=request_params, user=user) 2165 if not text: 2166 return None 2167 2168 if output == "raw": 2169 return text 2170 if output == "lines": 2171 return self._parse_lines(text) 2172 if output == "key_value": 2173 return self._parse_key_value(text) 2174 if output == "attributes": 2175 return self._parse_attributes(text) 2176 if output == "csv_dict": 2177 return self._parse_csv(text, header=True) 2178 return self._parse_csv(text) 2179 2180 # --------------------------------------------------------------------------- 2181 # Response parsers 2182 # 2183 # AKiPS replies in a handful of shapes. Each one is parsed in exactly one 2184 # place here, so the specific methods above and the generic call() cannot 2185 # drift apart in how they read the same reply. 2186 2187 @staticmethod 2188 def _parse_lines(text: str) -> list[str]: 2189 """ 2190 Split a reply into its non-blank lines. 2191 2192 Args: 2193 text (str): the raw reply from AKiPS 2194 Returns: 2195 A list of lines with blank ones removed 2196 """ 2197 return [line for line in text.split("\n") if line.strip()] 2198 2199 @staticmethod 2200 def _parse_key_value(text: str) -> dict[str, str]: 2201 """ 2202 Parse lines of '{key} = {value}', the shape mgroup replies in. 2203 2204 Args: 2205 text (str): the raw reply from AKiPS 2206 Returns: 2207 A dictionary of keys to their unsplit values 2208 """ 2209 data = {} 2210 for line in text.split("\n"): 2211 match = re.match(r"^(\S+)\s=\s(.*)$", line) 2212 if match: 2213 data[match.group(1)] = match.group(2) 2214 return data 2215 2216 @staticmethod 2217 def _parse_attributes(text: str) -> dict[str, dict[str, dict[str, str | None]]]: 2218 """ 2219 Parse lines of '{parent} {child} {attribute} = {value}', the shape 2220 mget replies in. An attribute with nothing after the equals has no 2221 value and is recorded as None. 2222 2223 Args: 2224 text (str): the raw reply from AKiPS 2225 Returns: 2226 A nested dictionary of parent, child, attribute to value 2227 """ 2228 data: dict[str, dict[str, dict[str, str | None]]] = {} 2229 unparsed = [] 2230 for line in text.split("\n"): 2231 match = re.match(r"^(\S+)\s(\S+)\s(\S+)\s=(\s(.*))?$", line) 2232 if not match: 2233 # Blank lines are how the reply ends and are not a problem. 2234 # Anything else is the server saying something this cannot 2235 # read, and it must not vanish: every method built on this one 2236 # would otherwise report less than AKiPS sent, with nothing to 2237 # say so. 2238 if line.strip(): 2239 unparsed.append(line) 2240 continue 2241 parent, child, attribute = ( 2242 match.group(1), 2243 match.group(2), 2244 match.group(3), 2245 ) 2246 data.setdefault(parent, {}).setdefault(child, {})[attribute] = match.group( 2247 5 2248 ) 2249 if unparsed: 2250 logger.warning( 2251 "Could not parse {} of {} attribute lines from akips, those " 2252 "values are missing from the result. First: {}".format( 2253 len(unparsed), 2254 len(unparsed) 2255 + sum( 2256 len(attributes) 2257 for children in data.values() 2258 for attributes in children.values() 2259 ), 2260 unparsed[0][:200], 2261 ) 2262 ) 2263 return data 2264 2265 @staticmethod 2266 def _parse_csv( 2267 text: str, fieldnames: list[str] | None = None, header: bool = False 2268 ) -> list[dict[str, str]] | list[list[str]]: 2269 """ 2270 Parse a CSV reply. AKiPS is not consistent about header rows, so the 2271 caller says which shape to expect rather than this guessing. 2272 2273 Args: 2274 text (str): the raw reply from AKiPS 2275 fieldnames (list): column names for a reply that carries no header 2276 header (bool): treat the first row as the header row 2277 Returns: 2278 A list of rows, as dictionaries when column names are known from 2279 either fieldnames or a header row, otherwise as lists 2280 """ 2281 buff = io.StringIO(text) 2282 if fieldnames is not None: 2283 return list(csv.DictReader(buff, fieldnames=fieldnames)) 2284 if header: 2285 return list(csv.DictReader(buff)) 2286 return [row for row in csv.reader(buff) if row] 2287 2288 # --------------------------------------------------------------------------- 2289 # Base operations 2290 2291 def _parse_enum(self, enum_string: str) -> dict[str, Any]: 2292 """ 2293 Attributes with a type of enum return five values separated by commas. 2294 2295 Args: 2296 enum_string (str): the raw enum string from AKiPS 2297 Returns: 2298 A dictionary with keys: number, value, created, modified, description 2299 Raises: 2300 AkipsError: if the provided string is not a valid enum type value 2301 """ 2302 # The trailing description is free text and routinely contains spaces, 2303 # so it takes the rest of the line rather than a non-whitespace run 2304 match = re.match(r"^(\S*),(\S*),(\S*),(\S*),(.*)$", enum_string) 2305 if match: 2306 entry = { 2307 "number": match.group(1), # list number (from MIB) 2308 "value": match.group(2), # text value (from MIB) 2309 # 'created': match.group(3), # time created (epoch timestamp) 2310 # 'modified': match.group(4), # time modified (epoch timestamp) 2311 "description": match.group(5), # child description 2312 } 2313 entry["created"] = datetime.fromtimestamp( 2314 int(match.group(3)), tz=pytz.timezone(self.server_timezone) 2315 ) 2316 entry["modified"] = datetime.fromtimestamp( 2317 int(match.group(4)), tz=pytz.timezone(self.server_timezone) 2318 ) 2319 return entry 2320 else: 2321 raise AkipsError(message=f"Not a ENUM type value: {enum_string}") 2322 2323 def _get_enum_attribute( 2324 self, 2325 attribute: str, 2326 child: str = "*", 2327 values: tuple[str, ...] | list[str] | None = None, 2328 group_filter: str = "any", 2329 groups: list[str] | None = None, 2330 ) -> dict[str, dict[str, Any]] | None: 2331 """ 2332 Pull one enum typed attribute and return it parsed, keyed by device. 2333 2334 Shared by the methods that ask 'which devices are in a bad state', in 2335 which the interesting answer is the enum's text value and when it last 2336 changed. Filtering by value is done by AKiPS rather than here, so a 2337 fleet wide query does not fetch every device to discard most of them. 2338 2339 Supporting AKiPS command syntax: 2340 2341 mget {type} [{parent regex} [{child regex} [{attribute regex}]]] 2342 [descr {/regex/}] [value {text|integer|/regex/}] 2343 [profile {profile name}] [any|all|not group {group name} ...] 2344 2345 Args: 2346 attribute (str): the attribute to read 2347 child (str): child name or pattern to match (default: '*') 2348 values (list): only report these enum values, or None for all 2349 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 2350 groups (list): list of group names to filter by (if any) 2351 Returns: 2352 A dictionary of device names to the parsed enum with the device 2353 'name' and 'child' added, or None if nothing matched 2354 Raises: 2355 AkipsError: if the AKiPS server returns an error 2356 """ 2357 params = {"cmds": f"mget * * {child} {attribute}"} 2358 if values: 2359 # [value {text|/regex/|integer|ipaddr}] 2360 params["cmds"] += " value /{}/".format("|".join(values)) 2361 if groups: 2362 # [any|all|not group {group name} ...] 2363 params["cmds"] += f" {group_filter} group {' '.join(groups)}" 2364 text = self._get(params=params) 2365 if not text: 2366 return None 2367 2368 data: dict[str, dict[str, Any]] = {} 2369 unparsed = [] 2370 for parent, children in self._parse_attributes(text).items(): 2371 for child_name, attributes in children.items(): 2372 for value in attributes.values(): 2373 if value is None: 2374 continue 2375 try: 2376 entry = self._parse_enum(value) 2377 except AkipsError: 2378 # One device reporting something unexpected should not 2379 # cost the answer for every other device 2380 unparsed.append(f"{parent} {child_name} = {value}") 2381 continue 2382 entry["name"] = parent 2383 entry["child"] = child_name 2384 if parent in data: 2385 # Keyed by device, so a device reporting this on more 2386 # than one child would quietly lose all but one 2387 logger.warning( 2388 "{} reports {} on more than one child, " 2389 "keeping {!r} and discarding {!r}".format( 2390 parent, attribute, data[parent]["child"], child_name 2391 ) 2392 ) 2393 continue 2394 data[parent] = entry 2395 if unparsed: 2396 logger.warning( 2397 "Could not parse {} of {} {} values from akips, those devices " 2398 "are missing from the result. First: {}".format( 2399 len(unparsed), len(unparsed) + len(data), attribute, unparsed[0] 2400 ) 2401 ) 2402 if not data: 2403 return None 2404 logger.debug("Found {} devices reporting {}".format(len(data), attribute)) 2405 return data 2406 2407 def _credentials_for( 2408 self, section: str, user: str | None = None 2409 ) -> tuple[str, str]: 2410 """ 2411 Pick the AKiPS account a request should authenticate as. 2412 2413 Args: 2414 section (str): API section being called 2415 user (str): force an account, 'ro' or 'rw', for a section whose 2416 requirement is not known or differs from the usual one 2417 Returns: 2418 A tuple of the username and password to send 2419 Raises: 2420 AkipsCredentialError: if the account this call needs has no password 2421 ValueError: if user names an account that does not exist 2422 """ 2423 if self._account_override is not None: 2424 # A custom account stands in for both 2425 return self._account_override 2426 2427 required = user if user is not None else self.SECTION_USERS.get(section) 2428 if required in ("ro", "api-ro"): 2429 required = "api-ro" 2430 elif required in ("rw", "api-rw"): 2431 required = "api-rw" 2432 elif required is not None: 2433 raise ValueError( 2434 f"Unknown AKiPS account {required!r}, expected 'ro' or 'rw'" 2435 ) 2436 2437 if required is None: 2438 # Either account works here, so use the lesser privileged one 2439 if self.ro_password is not None: 2440 return ("api-ro", self.ro_password) 2441 return ("api-rw", str(self.rw_password)) 2442 2443 password = self.ro_password if required == "api-ro" else self.rw_password 2444 if password is None: 2445 argument = "ro_password" if required == "api-ro" else "rw_password" 2446 raise AkipsCredentialError( 2447 f"{section} requires the {required} account, but no {argument} " 2448 f"was given to AKIPS()" 2449 ) 2450 return (required, password) 2451 2452 # Substrings that mark a name as holding a credential, whether it is a 2453 # query parameter or an AKiPS attribute. Matching loosely is deliberate: 2454 # over redacting costs a value in a debug log, under redacting leaks one. 2455 SENSITIVE_KEYS = ("password", "pass", "token", "secret", "key", "community") 2456 """Substrings marking a request parameter or AKiPS attribute whose value 2457 is redacted from log output. Extend it to redact more.""" 2458 2459 @classmethod 2460 def _is_sensitive_key(cls, name: str) -> bool: 2461 """Whether a parameter or attribute name looks like it holds a credential.""" 2462 return any(s in name.lower() for s in cls.SENSITIVE_KEYS) 2463 2464 def _redact_sensitive_params(self, params: dict[str, Any]) -> dict[str, Any]: 2465 """Return a copy of params with sensitive keys redacted from logging output.""" 2466 return { 2467 k: ("****" if self._is_sensitive_key(k) else v) for k, v in params.items() 2468 } 2469 2470 def _redact_text(self, text: str, literals: bool = True) -> str: 2471 """ 2472 Remove credentials from arbitrary text before it is logged or raised. 2473 2474 Matching on the query parameter covers the value whatever it looks 2475 like once URL encoded. Replacing the passwords this client holds 2476 catches them appearing outside a query string, but is unsafe for text 2477 that carries device data: a short password would rewrite every 2478 innocent occurrence of the same characters. Pass literals=False for 2479 anything that is not a URL or an error message. 2480 2481 Args: 2482 text (str): text that may contain credentials 2483 literals (bool): also replace the configured passwords wherever 2484 they appear (default: True) 2485 Returns: 2486 The text with any credential replaced by '****' 2487 """ 2488 text = re.sub(r"((?:password|passwd|pass)=)[^&\s]*", r"\1****", text) 2489 2490 # AKiPS keeps SNMP credentials as ordinary device attributes, so a 2491 # reply to something as innocent as get_device carries the community 2492 # string and the v3 auth and priv passwords. 2493 def redact_attribute(match: "re.Match[str]") -> str: 2494 if self._is_sensitive_key(match.group(2)): 2495 return f"{match.group(1)}****" 2496 return match.group(0) 2497 2498 text = re.sub(r"^(\S+\s\S+\s(\S+)\s=\s).*$", redact_attribute, text, flags=re.M) 2499 2500 if literals: 2501 for secret in (self.password, self.ro_password, self.rw_password): 2502 if secret: 2503 text = text.replace(secret, "****") 2504 return text 2505 2506 def _scrub_exception(self, err: BaseException) -> None: 2507 """ 2508 Strip credentials from an exception and everything it chains to. 2509 2510 AKiPS authenticates by query string and requests puts the failing URL 2511 in its exception messages, so an untouched exception carries the 2512 password into any log line or traceback that renders it. 2513 2514 The whole chain has to be scrubbed, not just the exception raised. 2515 requests raises its own error from the urllib3 one that caused it, and 2516 that inner exception holds the same URL, in its message and in a url 2517 attribute. Anything rendering a full traceback renders the chain, so 2518 leaving it means the password reaches wherever tracebacks are kept. 2519 2520 Rewriting in place keeps each exception's type and traceback, which a 2521 caller may be relying on, while making the text safe. 2522 2523 Args: 2524 err (BaseException): the exception to scrub, modified in place 2525 along with its __cause__ and __context__ chain 2526 """ 2527 seen: set[int] = set() 2528 pending: list[BaseException | None] = [err] 2529 while pending: 2530 node = pending.pop() 2531 if node is None or id(node) in seen: 2532 continue 2533 seen.add(id(node)) 2534 2535 original = str(node) 2536 redacted = self._redact_text(original) 2537 if redacted != original: 2538 node.args = (redacted,) 2539 2540 # urllib3 keeps the URL as an attribute of its own, which no 2541 # amount of message rewriting reaches 2542 url = getattr(node, "url", None) 2543 if isinstance(url, str): 2544 try: 2545 node.url = self._redact_text(url) # type: ignore[attr-defined] 2546 except AttributeError: 2547 pass 2548 2549 # A requests HTTPError carries the response it came from, and its 2550 # url is the one that was fetched, credentials and all. Error 2551 # reporters read that separately from the message. 2552 response = getattr(node, "response", None) 2553 if response is not None and isinstance(getattr(response, "url", None), str): 2554 try: 2555 response.url = self._redact_text(response.url) 2556 except AttributeError: 2557 pass 2558 2559 pending.append(node.__cause__) 2560 pending.append(node.__context__) 2561 2562 def _get( 2563 self, 2564 section: str = "api-db", 2565 params: dict[str, Any] | None = None, 2566 user: str | None = None, 2567 timeout: int | None = None, 2568 ) -> str: 2569 """ 2570 Base HTTP request against the AKiPS server for web API calls. 2571 2572 Sent as a POST with the password in the body, so it never appears in 2573 the request URI. Set use_post=False on the client to send the older 2574 GET form instead, which puts the password in the query string. The 2575 name is kept from when this only did GET, because it is called in 2576 two dozen places and is not part of the public API. 2577 2578 Section options are individually enabled via the AKiPS Web API Settings page. 2579 api-availability : Availability, default off 2580 api-db : Config and Events, default off 2581 api-config-viewer : Config Viewer, default off 2582 api-http-log : HTTP Log, default off 2583 api-flow : NetFlow, default off 2584 api-flow-timeseries : NetFlow Time-series, default off 2585 api-script : Site Script Functions, default off 2586 api-spm : Switch Port Mapper, default off 2587 api-msg : Syslog and Traps, default off 2588 api-unused-interfaces : Unused Interface, default off 2589 2590 Args: 2591 section (str): API section to call (default: 'api-db') 2592 params (dict): dictionary of parameters to pass to the server 2593 user (str): force the 'ro' or 'rw' account for this request 2594 timeout (int): seconds to wait for this one request, overriding 2595 the client's timeout. For a call whose cost does not depend 2596 on the client's usual work, such as a delete 2597 Returns: 2598 text output from the server 2599 Raises: 2600 AkipsCredentialError: if the account this section needs has no 2601 password 2602 AkipsAuthenticationError: if AKiPS rejects the credentials 2603 AkipsSectionDisabledError: if the section is not enabled on the 2604 server 2605 AkipsError: for any other error the AKiPS server returns 2606 requests.exceptions.HTTPError: for HTTP error responses 2607 requests.exceptions.ConnectionError: for connection errors 2608 requests.exceptions.Timeout: for request timeouts 2609 requests.exceptions.RequestException: for HTTP request errors 2610 """ 2611 server_url = f"https://{self.server}/{section}" 2612 2613 if section not in self.SECTION_USERS and section not in self._unknown_sections: 2614 # Warned rather than refused: AKiPS may add sections, and call() 2615 # exists so that reaching one does not have to wait for a release 2616 # here. A typo lands here too, which is the point. 2617 self._unknown_sections.add(section) 2618 logger.warning( 2619 "Unknown AKiPS API section {!r}, continuing anyway in case " 2620 "this server offers one this release does not know about. " 2621 "Known sections: {}".format( 2622 section, ", ".join(sorted(self.SECTION_USERS)) 2623 ) 2624 ) 2625 2626 # Work on a copy so credentials are never written into the dictionary 2627 # the caller passed in, and so params is optional as documented 2628 params = dict(params or {}) 2629 username, password = self._credentials_for(section, user) 2630 params["username"] = username 2631 2632 # The password travels in a POST body unless the caller has turned 2633 # that off, so it stays out of the request URI and out of everything 2634 # that records one. Everything else stays in the query string either 2635 # way, which is the form AKiPS documents and the only one older 2636 # servers accept. 2637 request_timeout = timeout if timeout is not None else self.timeout 2638 # A section may refuse the default method, so the choice is per 2639 # section rather than per client. See SECTION_METHODS. 2640 section_method = self.SECTION_METHODS.get(section, "POST") 2641 post = self.use_post and section_method == "POST" 2642 if self.use_post and not post and section not in self._method_warned: 2643 self._method_warned.add(section) 2644 # Logged at info, not warning. It is worth being able to see, 2645 # but an operator cannot act on it — the server is what refuses 2646 # the POST — and a permanent warning on every client teaches 2647 # people to ignore warnings, including the actionable ones this 2648 # module raises about a missing site script. 2649 logger.info( 2650 "Sending {} as {} rather than POST, so its password travels " 2651 "in the query string. {} does not answer a POST on any " 2652 "server seen so far. Set AKIPS.SECTION_METHODS[{!r}] = " 2653 "'POST' on a server where that is fixed.".format( 2654 section, section_method, section, section 2655 ) 2656 ) 2657 method = "POST" if post else "GET" 2658 data: dict[str, str] | None = None 2659 if post: 2660 data = {"password": password} 2661 else: 2662 params["password"] = password 2663 2664 logger.debug("{} url: {}".format(method, server_url)) 2665 logger.debug( 2666 "{} params: {}".format(method, self._redact_sensitive_params(params)) 2667 ) 2668 2669 try: 2670 with warnings.catch_warnings(): 2671 if not self.verify: 2672 # Scoped to this request on purpose. Disabling urllib3 2673 # warnings globally would also silence them for every 2674 # other library in the calling application. Note that 2675 # the warnings filter is process wide while this block 2676 # runs, so a concurrent thread could miss a warning. 2677 warnings.simplefilter( 2678 "ignore", urllib3.exceptions.InsecureRequestWarning 2679 ) 2680 if post: 2681 r = self.session.post( 2682 server_url, 2683 params=params, 2684 data=data, 2685 verify=self.verify, 2686 timeout=request_timeout, 2687 ) 2688 else: 2689 r = self.session.get( 2690 server_url, 2691 params=params, 2692 verify=self.verify, 2693 timeout=request_timeout, 2694 ) 2695 r.raise_for_status() 2696 except requests.exceptions.RequestException as err: 2697 # One handler for every requests failure: HTTPError, 2698 # ConnectionError and Timeout are all RequestException, and each 2699 # was doing the same thing here. The exception is scrubbed before 2700 # it is logged or re-raised, because requests reports the URL it 2701 # was fetching and AKiPS puts the password in that URL. 2702 self._scrub_exception(err) 2703 logger.error("AKiPS request failed: {}".format(err)) 2704 raise 2705 2706 # AKiPS can return a raw error message if something fails 2707 if re.match(r"^ERROR:", r.text): 2708 # Defense in depth: no AKiPS error seen so far echoes a credential 2709 # back, but this text goes into a log and an exception message. 2710 # Only the query parameter form is removed, never the password as 2711 # a literal, because a short one would rewrite matching characters 2712 # anywhere in the reply. 2713 message = self._redact_text(r.text, literals=False) 2714 logger.error("Web API request failed: {}".format(message)) 2715 # The two failures worth naming are the two that are nothing to do 2716 # with the call: the wrong password, and a section left switched 2717 # off. Both are ordinary first-run mistakes and both used to 2718 # arrive as an AkipsError saying only what AKiPS said. 2719 # 2720 # Matched loosely and on the distinctive phrase alone. AKiPS 2721 # prefixes the section name ('ERROR: api-db invalid 2722 # username/password') but that is not documented anywhere and 2723 # neither is the wording, so anything unrecognized has to keep 2724 # falling through to AkipsError rather than being forced into a 2725 # category. Both subclass AkipsError, so callers catching that 2726 # are unaffected. 2727 # Both carry what the call already knew, so a caller can act on 2728 # the section or the account without parsing AKiPS's prose. 2729 if re.search(r"invalid username/password", message, re.IGNORECASE): 2730 raise AkipsAuthenticationError( 2731 message=message, section=section, username=username 2732 ) 2733 if re.search(r"access is turned off", message, re.IGNORECASE): 2734 raise AkipsSectionDisabledError(message=message, section=section) 2735 raise AkipsError(message=message) 2736 else: 2737 logger.debug( 2738 "akips output: {}".format(self._redact_text(r.text, literals=False)) 2739 ) 2740 return r.text
A class to handle interactions with the AKiPS Web API
AKiPS ships two API accounts, api-ro and api-rw, and its sections do not all accept the same one. Supply the passwords for whichever accounts you need and each call uses the right one; see SECTION_USERS below for the mapping. A caller only reading data needs ro_password alone.
api = AKIPS('akips.example.com', ro_password='...', rw_password='...')
Four of the ten sections have methods of their own here: api-db, api-script, api-msg and api-availability. The rest are reached through call(), which sends a request to any section and parses the reply in the same shapes those methods use.
AKiPS stores data in three levels, a parent such as a device or user, then a child such as an interface or 'sys', then an attribute. What an attribute's value means depends on its type, per the AKiPS API guide:
counter always 1, so the value carries nothing
enum '{integer},{text}', e.g. '2,down'
gauge a scale factor, positive to multiply and negative to
divide, not a reading
integer a whole number, positive, negative or zero
RTT microseconds, not milliseconds
text up to 2000 characters
timestamp seconds since the Unix epoch
uptime seconds since the status last changed
Counters and gauges therefore come back from get_attributes() as their definition rather than a reading; the readings are in the time series database, which get_latest_values() and get_series() read.
Time filters are not all the same shape either. 'lastNm' and 'lastNh' are rolling windows, while 'lastNd' is calendar relative, so 'last1d' is today rather than 24 hours. AKiPS will say which it means, since 'tf' is one of the commands the read only account can run:
api.call('tf span last24h')
api.call('tf dump last1d')
AKIPS( server: str, username: str = 'api-ro', password: str | None = None, verify: bool | str = True, timezone: str = 'America/New_York', timeout: int = 30, ro_password: str | None = None, rw_password: str | None = None, use_post: bool = True)
137 def __init__( 138 self, 139 server: str, 140 username: str = "api-ro", 141 password: str | None = None, 142 verify: bool | str = True, 143 timezone: str = "America/New_York", 144 timeout: int = 30, 145 ro_password: str | None = None, 146 rw_password: str | None = None, 147 use_post: bool = True, 148 ) -> None: 149 self.server = server 150 """The AKiPS server hostname or IP address.""" 151 self.username = username 152 """Kept for callers who set it directly. With 'api-ro' or 'api-rw' 153 the password given alongside fills that account; with any other name 154 that pair is used for every section, which is how to use a custom 155 AKiPS API account.""" 156 self.password = password 157 """The password paired with username.""" 158 self.ro_password = ro_password 159 """Password for the api-ro account.""" 160 self.rw_password = rw_password 161 """Password for the api-rw account.""" 162 self.verify = verify 163 """Whether to verify TLS certificates. A path to a CA bundle can be 164 given instead, which is how to trust a server whose chain is missing 165 an intermediate without turning verification off entirely.""" 166 self.server_timezone = timezone 167 """Timezone of the AKiPS server, used to read the epochs it sends.""" 168 self.timeout = timeout 169 """HTTP timeout in seconds applied to every call. Assign to it to 170 change the timeout of an existing client, e.g. api.timeout = 60.""" 171 self.use_post = use_post 172 """Whether to send the password in a POST body instead of the query 173 string. True by default, and it should stay that way: URLs are 174 recorded by web servers, proxies and load balancers in their access 175 logs, and appear in exception messages and client history. A request 176 body is not logged that way, and a credential does not belong in a URL. 177 178 Set it to False only for a server that will not accept the POST form, 179 which puts the password back in the URL. Nothing falls back on its 180 own, because a silent retry over GET would leak the password at 181 exactly the moment the server turned out not to support this.""" 182 self.session = requests.Session() 183 """The requests session every call is made through.""" 184 # Sections warned about already, so a caller legitimately using a 185 # section this release does not know about is told once rather 186 # than on every call 187 self._unknown_sections: set[str] = set() 188 # Sections already warned about for falling back to GET, so a poll 189 # loop is told once rather than on every call 190 self._method_warned: set[str] = set() 191 192 # A username other than the two built in accounts is used for every 193 # section. AKiPS does not offer custom API accounts yet, but this is 194 # where they will land, and it keeps working for anyone already 195 # passing username and password directly. 196 self._account_override: tuple[str, str] | None = None 197 if password is not None: 198 if username == "api-ro" and self.ro_password is None: 199 self.ro_password = password 200 elif username == "api-rw" and self.rw_password is None: 201 self.rw_password = password 202 elif username not in ("api-ro", "api-rw"): 203 self._account_override = (username, password) 204 205 if ( 206 self._account_override is None 207 and self.ro_password is None 208 and self.rw_password is None 209 ): 210 raise AkipsCredentialError( 211 "No AKiPS password provided. Pass ro_password, rw_password, " 212 "or a username and password pair." 213 )
SECTION_USERS: dict[str, str | None] =
{'api-availability': 'api-ro', 'api-config-viewer': 'api-ro', 'api-db': None, 'api-flow': 'api-ro', 'api-flow-timeseries': 'api-ro', 'api-http-log': 'api-ro', 'api-msg': 'api-ro', 'api-script': 'api-rw', 'api-spm': 'api-ro', 'api-unused-interfaces': 'api-ro'}
Every API section AKiPS publishes, mapped to the account it accepts. None marks a section taking either, where the read only account is preferred. Also the list of sections known to exist.
SECTION_METHODS: dict[str, str] =
{'api-script': 'GET'}
HTTP method to use per section, for the sections that cannot take the default. Anything absent here is sent as POST when use_post is on.
api-script does not answer a POST. The server returns 200 headers in about a quarter of a second, then sends no body and holds the connection open until the client gives up, so every site script call hangs. The same call as GET returns normally, and api-db takes a POST with the identical header, so it is api-script specifically. Reported to AKiPS 2026-08-21.
The cost is that those calls put the password back in the query string, which is what use_post exists to prevent. It applies to get_device_by_ip(), set_group_membership() and delete_device(), and it is api-rw for two of them. GET is not a preference: it is what the section answers, and there is no third option, since the alternative is a call that never returns.
Nothing here assumes that will change. Sending the password in a POST body is itself undocumented — AKiPS support gave it out rather than the API guide describing it — so what any given server accepts is a question for that server rather than something this module can predict. This is a class attribute for that reason: if a server does take a POST on a section, say so without waiting for a release here.
AKIPS.SECTION_METHODS["api-script"] = "POST"
That is class wide and affects every client in the process.
username
Kept for callers who set it directly. With 'api-ro' or 'api-rw' the password given alongside fills that account; with any other name that pair is used for every section, which is how to use a custom AKiPS API account.
verify
Whether to verify TLS certificates. A path to a CA bundle can be given instead, which is how to trust a server whose chain is missing an intermediate without turning verification off entirely.
timeout
HTTP timeout in seconds applied to every call. Assign to it to change the timeout of an existing client, e.g. api.timeout = 60.
use_post
Whether to send the password in a POST body instead of the query string. True by default, and it should stay that way: URLs are recorded by web servers, proxies and load balancers in their access logs, and appear in exception messages and client history. A request body is not logged that way, and a credential does not belong in a URL.
Set it to False only for a server that will not accept the POST form, which puts the password back in the URL. Nothing falls back on its own, because a silent retry over GET would leak the password at exactly the moment the server turned out not to support this.
def
get_devices( self, group_filter: str = 'any', groups: list[str] | None = None) -> dict[str, dict[str, str | None]] | None:
220 def get_devices( 221 self, group_filter: str = "any", groups: list[str] | None = None 222 ) -> dict[str, dict[str, str | None]] | None: 223 """ 224 Pull a list of all devices and six key attributes of each, optionally 225 filtered by group membership. 226 227 This reads the 'sys' child and nothing else, and asks it for six 228 attributes: ip4addr, SNMPv2-MIB.sysName, SNMPv2-MIB.sysDescr, 229 SNMPv2-MIB.sysObjectID, SNMPv2-MIB.sysLocation and 230 SNMPv2-MIB.sysContact. Those are the values the AKiPS device edit page 231 shows read only, being what SNMP reported rather than what an operator 232 set, plus the address. 233 234 Both the child and the six are fixed here rather than arguments, 235 because this is the inventory view: every device comes back carrying 236 all six, as None where it reported no value, so they can be listed or 237 tabulated without checking each key first. Anything else the server 238 returns for a device is kept alongside them rather than dropped. 239 240 sysObjectID is worth knowing about: it identifies the model, such as 241 'ARUBA-MIB.ap225', which is often the field an inventory actually wants 242 and is more reliably populated than sysLocation. 243 244 For other attributes, other children, or a device's whole contents, 245 see get_attributes() and get_device(). 246 247 Because it asks for one child, this is the only method returning 248 attributes that does not keep the child level; the result is flattened 249 to device and attribute, which is the shape a listing wants. Should a 250 reply ever carry more than one child, their attributes are merged and 251 the last one read wins, where get_attributes() would keep them apart. 252 253 Supporting AKiPS command syntax: 254 255 mget {type} [{parent regex} [{child regex} [{attribute regex}]]] 256 [descr {/regex/}] [value {text|integer|/regex/}] 257 [profile {profile name}] [any|all|not group {group name} ...] 258 259 Args: 260 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 261 groups (list): list of group names to filter by (if any) 262 Returns: 263 A dictionary of device names to attribute dictionaries, or None if no devices found 264 Raises: 265 AkipsError: if the AKiPS server returns an error 266 """ 267 # The polled values the AKiPS device edit page shows read only, plus 268 # the address. Keeping to that set is deliberate: they are the 269 # standard SNMP system group fields an operator already recognizes. 270 attributes = [ 271 "ip4addr", 272 "SNMPv2-MIB.sysName", 273 "SNMPv2-MIB.sysDescr", 274 "SNMPv2-MIB.sysObjectID", 275 "SNMPv2-MIB.sysLocation", 276 "SNMPv2-MIB.sysContact", 277 ] 278 cmd_attributes = "|".join(attributes) 279 params = { 280 "cmds": f"mget text * sys /{cmd_attributes}/", 281 } 282 if groups: 283 # [any|all|not group {group name} ...] 284 group_list = " ".join(groups) 285 params["cmds"] += f" {group_filter} group {group_list}" 286 text = self._get(params=params) 287 if text: 288 data: dict[str, dict[str, str | None]] = {} 289 for parent, children in self._parse_attributes(text).items(): 290 # Every requested attribute is present, as None where the 291 # device reported no value for it 292 entry: dict[str, str | None] = dict.fromkeys(attributes) 293 for child_attributes in children.values(): 294 entry.update(child_attributes) 295 data[parent] = entry 296 # A reply that parses to nothing is nothing found, the same 297 # answer an empty reply gives, rather than an empty container 298 if not data: 299 return None 300 logger.debug("Found {} devices in akips".format(len(data.keys()))) 301 return data 302 return None
Pull a list of all devices and six key attributes of each, optionally filtered by group membership.
This reads the 'sys' child and nothing else, and asks it for six attributes: ip4addr, SNMPv2-MIB.sysName, SNMPv2-MIB.sysDescr, SNMPv2-MIB.sysObjectID, SNMPv2-MIB.sysLocation and SNMPv2-MIB.sysContact. Those are the values the AKiPS device edit page shows read only, being what SNMP reported rather than what an operator set, plus the address.
Both the child and the six are fixed here rather than arguments, because this is the inventory view: every device comes back carrying all six, as None where it reported no value, so they can be listed or tabulated without checking each key first. Anything else the server returns for a device is kept alongside them rather than dropped.
sysObjectID is worth knowing about: it identifies the model, such as 'ARUBA-MIB.ap225', which is often the field an inventory actually wants and is more reliably populated than sysLocation.
For other attributes, other children, or a device's whole contents, see get_attributes() and get_device().
Because it asks for one child, this is the only method returning attributes that does not keep the child level; the result is flattened to device and attribute, which is the shape a listing wants. Should a reply ever carry more than one child, their attributes are merged and the last one read wins, where get_attributes() would keep them apart.
Supporting AKiPS command syntax:
mget {type} [{parent regex} [{child regex} [{attribute regex}]]] [descr {/regex/}] [value {text|integer|/regex/}] [profile {profile name}] [any|all|not group {group name} ...]
Arguments:
- group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any')
- groups (list): list of group names to filter by (if any)
Returns:
A dictionary of device names to attribute dictionaries, or None if no devices found
Raises:
- AkipsError: if the AKiPS server returns an error
def
get_device(self, device: str) -> dict[str, dict[str, str | None]] | None:
304 def get_device(self, device: str) -> dict[str, dict[str, str | None]] | None: 305 """ 306 Pull all configuration attributes for a single device. The name is the 307 device's AKiPS name, its one primary key, which is either its sysName 308 or its IP address depending on how the server is set to name devices. 309 It is assigned at discovery, but a server can be told to reassign 310 devices already discovered from the other source, and an operator can 311 change one by hand, so a caller storing these as identifiers of its own 312 should not assume they never change. A device keyed by name still 313 carries its address as an attribute, and get_device_by_ip() resolves 314 an address back to the key. 315 316 This is the deep dive: every child and attribute this device holds, 317 which varies by device type. For the same fields across every device, 318 see get_devices(). 319 320 The result is the one device's children and their attributes, not a 321 dictionary keyed by the name that was just passed in: 322 323 device = api.get_device('TH840-A') 324 device['sys']['ip4addr'] 325 326 get_attributes() is the same query without that assumption, and keys 327 its result by device because it can match several. 328 329 Supporting AKiPS command syntax: 330 331 mget {type} [{parent regex} [{child regex} [{attribute regex}]]] 332 [descr {/regex/}] [value {text|integer|/regex/}] 333 [profile {profile name}] [any|all|not group {group name} ...] 334 335 Args: 336 device (str): the AKiPS name of one device, exactly. AKiPS matches 337 a bare name in this position exactly, so this is unambiguous; a 338 '/regex/' is refused, since matching several devices is what 339 get_attributes() is for 340 Returns: 341 A dictionary of the device's child names to attribute names and 342 values, or None if the device was not found 343 Raises: 344 ValueError: if a pattern is given, or if the reply somehow held 345 more than one device 346 AkipsError: if the AKiPS server returns an error 347 """ 348 # A bare name matches exactly in the parent position, but a '/regex/' 349 # does not, and this method has nowhere to put a second device. 350 if device.startswith("/") and device.endswith("/") and len(device) > 1: 351 raise ValueError( 352 "get_device takes one device name, not a pattern. Use " 353 "get_attributes(device={!r}) to match several".format(device) 354 ) 355 # get_attributes() with the filters left at their defaults, rather than 356 # building the same command a second time. 357 data = self.get_attributes(device=device) 358 if not data: 359 return None 360 if len(data) > 1: 361 raise ValueError( 362 "get_device matched {} devices ({}). Use get_attributes() " 363 "for more than one".format(len(data), ", ".join(sorted(data))) 364 ) 365 return next(iter(data.values()))
Pull all configuration attributes for a single device. The name is the device's AKiPS name, its one primary key, which is either its sysName or its IP address depending on how the server is set to name devices. It is assigned at discovery, but a server can be told to reassign devices already discovered from the other source, and an operator can change one by hand, so a caller storing these as identifiers of its own should not assume they never change. A device keyed by name still carries its address as an attribute, and get_device_by_ip() resolves an address back to the key.
This is the deep dive: every child and attribute this device holds, which varies by device type. For the same fields across every device, see get_devices().
The result is the one device's children and their attributes, not a dictionary keyed by the name that was just passed in:
device = api.get_device('TH840-A')
device['sys']['ip4addr']
get_attributes() is the same query without that assumption, and keys its result by device because it can match several.
Supporting AKiPS command syntax:
mget {type} [{parent regex} [{child regex} [{attribute regex}]]] [descr {/regex/}] [value {text|integer|/regex/}] [profile {profile name}] [any|all|not group {group name} ...]
Arguments:
- device (str): the AKiPS name of one device, exactly. AKiPS matches a bare name in this position exactly, so this is unambiguous; a '/regex/' is refused, since matching several devices is what get_attributes() is for
Returns:
A dictionary of the device's child names to attribute names and values, or None if the device was not found
Raises:
- ValueError: if a pattern is given, or if the reply somehow held more than one device
- AkipsError: if the AKiPS server returns an error
def
get_unreachable( self, children: str = 'ping4|ping6|sys') -> dict[str, dict[str, typing.Any]] | None:
377 def get_unreachable( 378 self, children: str = UNREACHABLE_CHILDREN 379 ) -> dict[str, dict[str, Any]] | None: 380 """ 381 Pull a list of unreachable devices by Ping and SNMP state. 382 383 Supporting AKiPS command syntax: 384 385 mget {type} [{parent regex} [{child regex} [{attribute regex}]]] 386 [descr {/regex/}] [value {text|integer|/regex/}] 387 [profile {profile name}] [any|all|not group {group name} ...] 388 389 Note the {type} field is left off here. These are enum attributes, so 390 narrowing with 'mget text' returns no rows at all rather than an 391 error, even though it is the obvious thing to reach for. 392 393 **Asking only for what is broken is not necessarily the cheap way.** 394 On one fleet this call took roughly three times as long as 395 get_ping_state(), while returning 289 times fewer rows: filtering on 396 value makes AKiPS match every device's enum rather than dump the 397 attribute, and this asks for two attributes where that asks for one. 398 Whether that holds elsewhere is unknown, and it may be a property of 399 that server's data rather than of the query. It is recorded because 400 the opposite is the natural assumption: if this call is hot, measure 401 it against reading the state outright and filtering here. 402 403 Args: 404 children (str): regex of children to search, defaulting to the 405 ones AKiPS reports these attributes under. Pass '*' for 406 every child of every device, which is correct for a site 407 naming them differently and considerably slower 408 Returns: 409 A dictionary of device names to their unreachable attributes, or 410 None if nothing was reported as down 411 Raises: 412 AkipsError: if the AKiPS server returns an error 413 """ 414 # '*' is the wildcard rather than a pattern, so it is the one value 415 # that must not be wrapped in slashes 416 child = children if children == "*" else f"/{children}/" 417 params = { 418 "cmds": f"mget * * {child} /PING.icmpState|SNMP.snmpState/ value /down/", 419 } 420 text = self._get(params=params) 421 if text: 422 data: dict[str, dict[str, Any]] = {} 423 unparsed = [] 424 lines = text.split("\n") 425 for line in lines: 426 match = re.match( 427 r"^(\S+)\s(\S+)\s(\S+)\s=\s(\S+),(\S+),(\S+),(\S+),(\S+)?$", line 428 ) 429 if not match: 430 if line.strip(): 431 # A line reporting a device down that this does not 432 # understand must not vanish: under reporting an 433 # outage is the worst thing this call can do. 434 unparsed.append(line) 435 continue 436 # epoch fields are in the server's timezone 437 name = match.group(1) 438 attribute = match.group(3) 439 event_start = datetime.fromtimestamp( 440 int(match.group(7)), tz=pytz.timezone(self.server_timezone) 441 ) 442 device_added = datetime.fromtimestamp( 443 int(match.group(6)), tz=pytz.timezone(self.server_timezone) 444 ) 445 if name not in data: 446 # populate a starting point for this device 447 data[name] = { 448 "name": name, 449 "ping_state": "n/a", 450 "snmp_state": "n/a", 451 "event_start": event_start, # epoch in local timezone 452 } 453 if attribute == "PING.icmpState": 454 data[name]["ping_state"] = match.group(5) 455 # A device down on both checks reports one child, index 456 # and address. Ping wins them, because it is the only 457 # line carrying an address, and assigning here while the 458 # SNMP branch below only fills gaps makes the result the 459 # same whichever order the lines arrive in. 460 data[name]["child"] = match.group(2) 461 data[name]["index"] = match.group(4) 462 data[name]["device_added"] = device_added 463 data[name]["ip4addr"] = match.group(8) 464 elif attribute == "SNMP.snmpState": 465 data[name]["snmp_state"] = match.group(5) 466 data[name].setdefault("child", match.group(2)) 467 data[name].setdefault("index", match.group(4)) 468 data[name].setdefault("device_added", device_added) 469 data[name].setdefault("ip4addr", None) 470 # A device down on both ping and SNMP has two start times; the 471 # outage began at the earlier of them. This has to be the only 472 # place event_start is set, or the comparison is against the 473 # value just written from this same line and the last line seen 474 # would always win. 475 if event_start < data[name]["event_start"]: 476 data[name]["event_start"] = event_start 477 if unparsed: 478 logger.warning( 479 "Could not parse {} of {} unreachable lines from akips, " 480 "those devices are missing from the result. First: {}".format( 481 len(unparsed), len(unparsed) + len(data), unparsed[0][:200] 482 ) 483 ) 484 # A reply that parses to nothing is nothing found, the same 485 # answer an empty reply gives, rather than an empty container 486 if not data: 487 return None 488 logger.debug("Found {} devices in akips".format(len(data))) 489 return data 490 return None
Pull a list of unreachable devices by Ping and SNMP state.
Supporting AKiPS command syntax:
mget {type} [{parent regex} [{child regex} [{attribute regex}]]] [descr {/regex/}] [value {text|integer|/regex/}] [profile {profile name}] [any|all|not group {group name} ...]
Note the {type} field is left off here. These are enum attributes, so narrowing with 'mget text' returns no rows at all rather than an error, even though it is the obvious thing to reach for.
Asking only for what is broken is not necessarily the cheap way. On one fleet this call took roughly three times as long as get_ping_state(), while returning 289 times fewer rows: filtering on value makes AKiPS match every device's enum rather than dump the attribute, and this asks for two attributes where that asks for one. Whether that holds elsewhere is unknown, and it may be a property of that server's data rather than of the query. It is recorded because the opposite is the natural assumption: if this call is hot, measure it against reading the state outright and filtering here.
Arguments:
- children (str): regex of children to search, defaulting to the ones AKiPS reports these attributes under. Pass '*' for every child of every device, which is correct for a site naming them differently and considerably slower
Returns:
A dictionary of device names to their unreachable attributes, or None if nothing was reported as down
Raises:
- AkipsError: if the AKiPS server returns an error
def
get_ping_state( self, states: tuple[str, ...] | list[str] | None = None, child: str = 'ping4', group_filter: str = 'any', groups: list[str] | None = None) -> dict[str, dict[str, typing.Any]] | None:
492 def get_ping_state( 493 self, 494 states: tuple[str, ...] | list[str] | None = None, 495 child: str = "ping4", 496 group_filter: str = "any", 497 groups: list[str] | None = None, 498 ) -> dict[str, dict[str, Any]] | None: 499 """ 500 Pull the ping state of every device, with when it last changed. 501 502 get_unreachable() answers 'what is broken now' and so asks only for 503 the devices that are down. This asks the same record without that 504 filter, which is what to call for a device that is up: its state, and 505 both of the epochs the enum carries. 506 507 **The two epochs are the useful part and their names undersell them.** 508 'created' is when AKiPS started polling the device, so it is the date 509 the device was added to AKiPS, and it is not otherwise reachable for a 510 device that is healthy. 'modified' is the instant the state last 511 changed, not a row-touched timestamp. Both arrive as aware datetimes 512 in the server's timezone rather than as the integers the raw attribute 513 holds, so nothing needs converting. 514 515 Those two answer the column AKiPS shows on its own device dashboard, 516 the one reading Uptime on a device that is up and Downtime on a device 517 that is down. It is not sysUpTime: the figure is now minus 'modified' 518 and 'value' decides which word. sysUpTime counts from the last boot 519 and keeps counting through an outage, so the two disagree on exactly 520 the devices somebody is looking at. 521 522 get_snmp_state() is the same record for the SNMP agent. It answers 523 for far fewer devices, because AKiPS pings everything it holds and 524 polls SNMP only where SNMP is configured. 525 526 Args: 527 states (list): only report these states, e.g. ('down',), or None 528 for every device whatever its state, which is the default 529 child (str): which ping child to read (default: 'ping4'). Pass 530 'ping4|ping6' for both, though a device answering on both 531 keeps only one entry and warns, since this is keyed by device 532 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 533 groups (list): list of group names to filter by (if any) 534 Returns: 535 A dictionary of device names to the parsed state, or None if no 536 device matched. Each entry carries the enum fields described on 537 _parse_enum, plus the device 'name' and 'child' 538 Raises: 539 AkipsError: if the AKiPS server returns an error 540 """ 541 return self._get_enum_attribute( 542 "PING.icmpState", 543 child=child, 544 values=states, 545 group_filter=group_filter, 546 groups=groups, 547 )
Pull the ping state of every device, with when it last changed.
get_unreachable() answers 'what is broken now' and so asks only for the devices that are down. This asks the same record without that filter, which is what to call for a device that is up: its state, and both of the epochs the enum carries.
The two epochs are the useful part and their names undersell them. 'created' is when AKiPS started polling the device, so it is the date the device was added to AKiPS, and it is not otherwise reachable for a device that is healthy. 'modified' is the instant the state last changed, not a row-touched timestamp. Both arrive as aware datetimes in the server's timezone rather than as the integers the raw attribute holds, so nothing needs converting.
Those two answer the column AKiPS shows on its own device dashboard, the one reading Uptime on a device that is up and Downtime on a device that is down. It is not sysUpTime: the figure is now minus 'modified' and 'value' decides which word. sysUpTime counts from the last boot and keeps counting through an outage, so the two disagree on exactly the devices somebody is looking at.
get_snmp_state() is the same record for the SNMP agent. It answers for far fewer devices, because AKiPS pings everything it holds and polls SNMP only where SNMP is configured.
Arguments:
- states (list): only report these states, e.g. ('down',), or None for every device whatever its state, which is the default
- child (str): which ping child to read (default: 'ping4'). Pass 'ping4|ping6' for both, though a device answering on both keeps only one entry and warns, since this is keyed by device
- group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any')
- groups (list): list of group names to filter by (if any)
Returns:
A dictionary of device names to the parsed state, or None if no device matched. Each entry carries the enum fields described on _parse_enum, plus the device 'name' and 'child'
Raises:
- AkipsError: if the AKiPS server returns an error
def
get_snmp_state( self, states: tuple[str, ...] | list[str] | None = None, child: str = 'sys', group_filter: str = 'any', groups: list[str] | None = None) -> dict[str, dict[str, typing.Any]] | None:
549 def get_snmp_state( 550 self, 551 states: tuple[str, ...] | list[str] | None = None, 552 child: str = "sys", 553 group_filter: str = "any", 554 groups: list[str] | None = None, 555 ) -> dict[str, dict[str, Any]] | None: 556 """ 557 Pull the SNMP agent state of every SNMP polled device. 558 559 The SNMP counterpart to get_ping_state(), reading the same kind of 560 record with the same fields: the state, when the device was added, 561 and when the state last changed. 562 563 **It answers for fewer devices than get_ping_state() does, and the 564 difference is large.** AKiPS pings everything it holds but polls SNMP 565 only where SNMP is configured, so a device absent from this result is 566 usually one that is not SNMP polled rather than one whose agent has 567 stopped answering. Measured on one fleet, 5,821 devices of 16,785 568 appeared here and the rest were ICMP only. A caller that assumes the 569 same denominator as the ping call reads two thirds of the fleet as 570 broken. 571 572 Absence therefore means unknown, not down. The devices answering here 573 are the same ones answering SNMPv2-MIB.sysUpTime, which is a way to 574 confirm the denominator on a given server. 575 576 Args: 577 states (list): only report these states, e.g. ('down',), or None 578 for every SNMP polled device, which is the default 579 child (str): which child holds the agent state (default: 'sys') 580 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 581 groups (list): list of group names to filter by (if any) 582 Returns: 583 A dictionary of device names to the parsed state, or None if no 584 device matched. Each entry carries the enum fields described on 585 _parse_enum, plus the device 'name' and 'child' 586 Raises: 587 AkipsError: if the AKiPS server returns an error 588 """ 589 return self._get_enum_attribute( 590 "SNMP.snmpState", 591 child=child, 592 values=states, 593 group_filter=group_filter, 594 groups=groups, 595 )
Pull the SNMP agent state of every SNMP polled device.
The SNMP counterpart to get_ping_state(), reading the same kind of record with the same fields: the state, when the device was added, and when the state last changed.
It answers for fewer devices than get_ping_state() does, and the difference is large. AKiPS pings everything it holds but polls SNMP only where SNMP is configured, so a device absent from this result is usually one that is not SNMP polled rather than one whose agent has stopped answering. Measured on one fleet, 5,821 devices of 16,785 appeared here and the rest were ICMP only. A caller that assumes the same denominator as the ping call reads two thirds of the fleet as broken.
Absence therefore means unknown, not down. The devices answering here are the same ones answering SNMPv2-MIB.sysUpTime, which is a way to confirm the denominator on a given server.
Arguments:
- states (list): only report these states, e.g. ('down',), or None for every SNMP polled device, which is the default
- child (str): which child holds the agent state (default: 'sys')
- group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any')
- groups (list): list of group names to filter by (if any)
Returns:
A dictionary of device names to the parsed state, or None if no device matched. Each entry carries the enum fields described on _parse_enum, plus the device 'name' and 'child'
Raises:
- AkipsError: if the AKiPS server returns an error
def
get_attributes( self, device: str = '*', child: str = '*', attribute: str = '*', value: str | None = None, group_filter: str = 'any', groups: list[str] | None = None) -> dict[str, dict[str, dict[str, str | None]]] | None:
597 def get_attributes( 598 self, 599 device: str = "*", 600 child: str = "*", 601 attribute: str = "*", 602 value: str | None = None, 603 group_filter: str = "any", 604 groups: list[str] | None = None, 605 ) -> dict[str, dict[str, dict[str, str | None]]] | None: 606 """ 607 Pull attribute values with variable search criteria. Search criteria defaults to 608 a wildcard match but can be filtered by 'device' name or pattern, 'child' name or pattern, 609 'attribute' name or pattern, and/or attribute 'value' or pattern. Additionally, 610 results can be filtered by group membership using 'any', 'all', or 'not' operators 611 along with one or more group names. 612 613 Supporting AKiPS command syntax: 614 615 mget {type} [{parent regex} [{child regex} [{attribute regex}]]] 616 [descr {/regex/}] [value {text|integer|/regex/}] 617 [profile {profile name}] [any|all|not group {group name} ...] 618 619 **A bare attribute name matches exactly and matches nothing.** AKiPS 620 qualifies most attributes with their MIB, so 'sysUpTime' matches no 621 attribute anywhere while 'SNMPv2-MIB.sysUpTime' matches on every 622 device that reports it. The unqualified form is not an error: it 623 returns an empty result, which looks exactly like a fleet where 624 nothing reports that attribute. Use a pattern, '/sysUpTime/', when 625 the qualified name is not known. 626 627 Args: 628 device (str): device name or pattern to match (default: '*') 629 child (str): child name or pattern to match (default: '*') 630 attribute (str): attribute name or pattern to match (default: '*'). 631 A bare name must match exactly, MIB prefix included; see above 632 value (str): value or pattern to match (default: None) 633 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 634 groups (list): list of group names to filter by (if any) 635 Returns: 636 A nested dictionary of device names to child names to attribute names and values, 637 or None if no devices found 638 Raises: 639 AkipsError: if the AKiPS server returns an error 640 """ 641 params = { 642 "cmds": f"mget * {device} {child} {attribute}", 643 } 644 if value: 645 # [value {text|/regex/|integer|ipaddr}] 646 params["cmds"] += f" value {value}" 647 if groups: 648 # [any|all|not group {group name} ...] 649 group_list = " ".join(groups) 650 params["cmds"] += f" {group_filter} group {group_list}" 651 text = self._get(params=params) 652 if text: 653 data = self._parse_attributes(text) 654 # A reply that parses to nothing is nothing found, the same 655 # answer an empty reply gives, rather than an empty container 656 if not data: 657 return None 658 logger.debug("Found {} devices in akips".format(len(data.keys()))) 659 return data 660 return None
Pull attribute values with variable search criteria. Search criteria defaults to a wildcard match but can be filtered by 'device' name or pattern, 'child' name or pattern, 'attribute' name or pattern, and/or attribute 'value' or pattern. Additionally, results can be filtered by group membership using 'any', 'all', or 'not' operators along with one or more group names.
Supporting AKiPS command syntax:
mget {type} [{parent regex} [{child regex} [{attribute regex}]]] [descr {/regex/}] [value {text|integer|/regex/}] [profile {profile name}] [any|all|not group {group name} ...]
A bare attribute name matches exactly and matches nothing. AKiPS qualifies most attributes with their MIB, so 'sysUpTime' matches no attribute anywhere while 'SNMPv2-MIB.sysUpTime' matches on every device that reports it. The unqualified form is not an error: it returns an empty result, which looks exactly like a fleet where nothing reports that attribute. Use a pattern, '/sysUpTime/', when the qualified name is not known.
Arguments:
- device (str): device name or pattern to match (default: '*')
- child (str): child name or pattern to match (default: '*')
- attribute (str): attribute name or pattern to match (default: '*'). A bare name must match exactly, MIB prefix included; see above
- value (str): value or pattern to match (default: None)
- group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any')
- groups (list): list of group names to filter by (if any)
Returns:
A nested dictionary of device names to child names to attribute names and values, or None if no devices found
Raises:
- AkipsError: if the AKiPS server returns an error
UPS_ABNORMAL_OUTPUT_SOURCES =
('other', 'none', 'bypass', 'battery', 'booster', 'reducer')
Output sources get_ups_output_source() reports by default, being every UPS-MIB source but normal, so every state that is not running on mains.
UPS_ABNORMAL_BATTERY_STATES =
('unknown', 'batteryLow', 'batteryDepleted')
Battery states get_ups_battery_status() reports by default, being every UPS-MIB state but batteryNormal.
LIEBERT_BATTERY_TEST_ATTRIBUTE =
'LIEBERT-GP-POWER-MIB.lgpPwrBatteryTestResult'
The attribute get_liebert_battery_test() reads. Battery test results are not in the standard UPS-MIB, so this one is vendor specific.
def
get_ups_battery_status( self, states: tuple[str, ...] | list[str] | None = ('unknown', 'batteryLow', 'batteryDepleted'), group_filter: str = 'any', groups: list[str] | None = None) -> dict[str, dict[str, typing.Any]] | None:
697 def get_ups_battery_status( 698 self, 699 states: tuple[str, ...] | list[str] | None = UPS_ABNORMAL_BATTERY_STATES, 700 group_filter: str = "any", 701 groups: list[str] | None = None, 702 ) -> dict[str, dict[str, Any]] | None: 703 """ 704 Pull the UPS devices whose battery is not reporting as normal. 705 706 UPS-MIB reports the battery's own condition, separately from where the 707 UPS is drawing its output, which get_ups_output_source() reads. By 708 default this returns only the states other than batteryNormal. 709 710 This is the battery's condition, not how long it would last. AKiPS 711 keeps the numeric readings such as upsEstimatedMinutesRemaining in its 712 time series database rather than alongside these, so they come from 713 get_series() rather than from here. Reading them with mget returns 714 the gauge's scaling factor, which is identical for every device. 715 716 Args: 717 states (list): battery states to report, defaulting to everything 718 except batteryNormal. Pass None for every UPS whatever its 719 battery state 720 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 721 groups (list): list of group names to filter by (if any) 722 Returns: 723 A dictionary of device names to the parsed state, or None if no 724 device matched. Each entry carries the enum fields described on 725 _parse_enum, where 'value' is the battery state and 'modified' is 726 when it last changed, plus the device 'name' and 'child' 727 Raises: 728 AkipsError: if the AKiPS server returns an error 729 """ 730 return self._get_enum_attribute( 731 "UPS-MIB.upsBatteryStatus", 732 child="battery", 733 values=states, 734 group_filter=group_filter, 735 groups=groups, 736 )
Pull the UPS devices whose battery is not reporting as normal.
UPS-MIB reports the battery's own condition, separately from where the UPS is drawing its output, which get_ups_output_source() reads. By default this returns only the states other than batteryNormal.
This is the battery's condition, not how long it would last. AKiPS keeps the numeric readings such as upsEstimatedMinutesRemaining in its time series database rather than alongside these, so they come from get_series() rather than from here. Reading them with mget returns the gauge's scaling factor, which is identical for every device.
Arguments:
- states (list): battery states to report, defaulting to everything except batteryNormal. Pass None for every UPS whatever its battery state
- group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any')
- groups (list): list of group names to filter by (if any)
Returns:
A dictionary of device names to the parsed state, or None if no device matched. Each entry carries the enum fields described on _parse_enum, where 'value' is the battery state and 'modified' is when it last changed, plus the device 'name' and 'child'
Raises:
- AkipsError: if the AKiPS server returns an error
def
get_ups_output_source( self, states: tuple[str, ...] | list[str] | None = ('other', 'none', 'bypass', 'battery', 'booster', 'reducer'), group_filter: str = 'any', groups: list[str] | None = None) -> dict[str, dict[str, typing.Any]] | None:
738 def get_ups_output_source( 739 self, 740 states: tuple[str, ...] | list[str] | None = UPS_ABNORMAL_OUTPUT_SOURCES, 741 group_filter: str = "any", 742 groups: list[str] | None = None, 743 ) -> dict[str, dict[str, Any]] | None: 744 """ 745 Pull the UPS devices that are not running on mains power. 746 747 UPS-MIB reports where a UPS is drawing its output from, which is 748 'normal' when all is well. By default this returns every other value, 749 so the result is the list of UPSes worth looking at. That includes 750 'none', a UPS delivering no output at all, and 'other', one that 751 cannot classify its own source. 752 753 Note this is the output source, not the battery's own health, which 754 UPS-MIB reports separately as upsBatteryStatus. 755 756 Args: 757 states (list): output sources to report, defaulting to everything 758 except 'normal'. Pass None for every UPS whatever its state 759 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 760 groups (list): list of group names to filter by (if any) 761 Returns: 762 A dictionary of device names to the parsed state, or None if no 763 device matched. Each entry carries the enum fields described on 764 _parse_enum, where 'value' is the output source and 'modified' is 765 when it last changed, plus the device 'name' and 'child' 766 Raises: 767 AkipsError: if the AKiPS server returns an error 768 """ 769 return self._get_enum_attribute( 770 "UPS-MIB.upsOutputSource", 771 # The child the UPS itself is reported under, as against 772 # 'battery' for the battery attributes. Naming it keeps AKiPS 773 # from walking every child of every device, which is most of the 774 # cost of the query. 775 child="ups", 776 values=states, 777 group_filter=group_filter, 778 groups=groups, 779 )
Pull the UPS devices that are not running on mains power.
UPS-MIB reports where a UPS is drawing its output from, which is 'normal' when all is well. By default this returns every other value, so the result is the list of UPSes worth looking at. That includes 'none', a UPS delivering no output at all, and 'other', one that cannot classify its own source.
Note this is the output source, not the battery's own health, which UPS-MIB reports separately as upsBatteryStatus.
Arguments:
- states (list): output sources to report, defaulting to everything except 'normal'. Pass None for every UPS whatever its state
- group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any')
- groups (list): list of group names to filter by (if any)
Returns:
A dictionary of device names to the parsed state, or None if no device matched. Each entry carries the enum fields described on _parse_enum, where 'value' is the output source and 'modified' is when it last changed, plus the device 'name' and 'child'
Raises:
- AkipsError: if the AKiPS server returns an error
def
get_liebert_battery_test( self, results: tuple[str, ...] | list[str] | None = ('failed',), attribute: str = 'LIEBERT-GP-POWER-MIB.lgpPwrBatteryTestResult', group_filter: str = 'any', groups: list[str] | None = None) -> dict[str, dict[str, typing.Any]] | None:
781 def get_liebert_battery_test( 782 self, 783 results: tuple[str, ...] | list[str] | None = ("failed",), 784 attribute: str = LIEBERT_BATTERY_TEST_ATTRIBUTE, 785 group_filter: str = "any", 786 groups: list[str] | None = None, 787 ) -> dict[str, dict[str, Any]] | None: 788 """ 789 Pull the results of the last battery self test on Liebert and Vertiv 790 UPS equipment. 791 792 By default this returns only the failures, which is the list of 793 batteries to replace. Pass results=None for every UPS and its last 794 result. 795 796 The vendor is in the name on purpose. Battery test results are not in 797 the standard UPS-MIB, so this reads an attribute only Liebert and 798 Vertiv equipment reports. Run against another vendor's fleet it 799 returns nothing, which would otherwise read as good news. Another 800 vendor's equivalent attribute can be passed to reuse the same parsing 801 and shape. 802 803 Args: 804 results (list): test results to report, defaulting to failures 805 only. Pass None for every UPS whatever its last result 806 attribute (str): the vendor attribute holding the result 807 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 808 groups (list): list of group names to filter by (if any) 809 Returns: 810 A dictionary of device names to the parsed result, or None if no 811 device matched. Each entry carries the enum fields described on 812 _parse_enum, where 'value' is the test result and 'modified' is 813 when it last changed, plus the device 'name' and 'child' 814 Raises: 815 AkipsError: if the AKiPS server returns an error 816 """ 817 return self._get_enum_attribute( 818 attribute, 819 child="battery", 820 values=results, 821 group_filter=group_filter, 822 groups=groups, 823 )
Pull the results of the last battery self test on Liebert and Vertiv UPS equipment.
By default this returns only the failures, which is the list of batteries to replace. Pass results=None for every UPS and its last result.
The vendor is in the name on purpose. Battery test results are not in the standard UPS-MIB, so this reads an attribute only Liebert and Vertiv equipment reports. Run against another vendor's fleet it returns nothing, which would otherwise read as good news. Another vendor's equivalent attribute can be passed to reuse the same parsing and shape.
Arguments:
- results (list): test results to report, defaulting to failures only. Pass None for every UPS whatever its last result
- attribute (str): the vendor attribute holding the result
- group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any')
- groups (list): list of group names to filter by (if any)
Returns:
A dictionary of device names to the parsed result, or None if no device matched. Each entry carries the enum fields described on _parse_enum, where 'value' is the test result and 'modified' is when it last changed, plus the device 'name' and 'child'
Raises:
- AkipsError: if the AKiPS server returns an error
def
get_group_membership( self, device: str = '*', group_filter: str = 'any', groups: list[str] | None = None) -> dict[str, list[str]] | None:
827 def get_group_membership( 828 self, 829 device: str = "*", 830 group_filter: str = "any", 831 groups: list[str] | None = None, 832 ) -> dict[str, list[str]] | None: 833 """ 834 Pull a list of device names to group memberships. Defaults to all devices 835 and all groups (including the special 'maintenance_mode' group). 836 837 Supporting AKiPS command syntax: 838 839 mgroup {type} [{parent regex}] 840 [any|all|not group {group name} ...] 841 842 Args: 843 device (str): device name or pattern to match (default: '*') 844 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 845 groups (list): list of group names to filter by (if any) 846 Returns: 847 A dictionary of device names to lists of group names, or None if no devices found 848 Raises: 849 AkipsError: if the AKiPS server returns an error 850 """ 851 params = { 852 "cmds": f"mgroup * {device}", 853 } 854 if groups: 855 group_list = " ".join(groups) 856 params["cmds"] += f" {group_filter} group {group_list}" 857 text = self._get(params=params) 858 if text: 859 data = { 860 device_name: groups_value.split(",") 861 for device_name, groups_value in self._parse_key_value(text).items() 862 } 863 # A reply that parses to nothing is nothing found, the same 864 # answer an empty reply gives, rather than an empty container 865 if not data: 866 return None 867 logger.debug( 868 "Found {} device and group mappings in akips".format(len(data.keys())) 869 ) 870 return data 871 return None
Pull a list of device names to group memberships. Defaults to all devices and all groups (including the special 'maintenance_mode' group).
Supporting AKiPS command syntax:
mgroup {type} [{parent regex}] [any|all|not group {group name} ...]
Arguments:
- device (str): device name or pattern to match (default: '*')
- group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any')
- groups (list): list of group names to filter by (if any)
Returns:
A dictionary of device names to lists of group names, or None if no devices found
Raises:
- AkipsError: if the AKiPS server returns an error
def
get_events( self, event_type: str = 'all', period: str = 'last1h', device: str = '*', child: str = '*', attribute: str = '*', group_filter: str = 'any', groups: list[str] | None = None) -> list[dict[str, str]] | None:
875 def get_events( 876 self, 877 event_type: str = "all", 878 period: str = "last1h", 879 device: str = "*", 880 child: str = "*", 881 attribute: str = "*", 882 group_filter: str = "any", 883 groups: list[str] | None = None, 884 ) -> list[dict[str, str]] | None: 885 """ 886 Pull a list of events over a time period with optional filtering by device, 887 child, attribute, and/or group membership. Defaults to all event types over 888 the last hour. Review AKiPS documentation for details on event types and 889 time filter syntax. 890 891 Supporting AKiPS command syntax: 892 893 mget event {all,critical,enum,threshold,uptime} 894 time {time filter} [{parent regex} {child regex} 895 {attribute regex}] [profile {profile name}] 896 [any|all|not group {group name} ...] 897 898 Args: 899 event_type (str): type of events to retrieve (default: 'all') 900 period (str): time period to retrieve events from (default: 'last1h') 901 device (str): device name or pattern to match (default: '*') 902 child (str): child name or pattern to match (default: '*') 903 attribute (str): attribute name or pattern to match (default: '*') 904 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 905 groups (list): list of group names to filter by (if any) 906 Returns: 907 A list of event dictionaries, or None if no events found 908 Raises: 909 AkipsError: if the AKiPS server returns an error 910 """ 911 params = { 912 "cmds": f"mget event {event_type} time {period} {device} {child} {attribute}" 913 } 914 if groups: 915 # [any|all|not group {group name} ...] 916 group_list = " ".join(groups) 917 params["cmds"] += f" {group_filter} group {group_list}" 918 text = self._get(params=params) 919 if text: 920 data = [] 921 lines = text.split("\n") 922 for line in lines: 923 match = re.match( 924 r"^(\S+)\s(\S+)\s(\S+)\s(\S+)\s(\S+)\s(\S+)\s(.*)$", line 925 ) 926 if match: 927 entry = { 928 "epoch": match.group(1), 929 "parent": match.group(2), 930 "child": match.group(3), 931 "attribute": match.group(4), 932 "type": match.group(5), 933 "flags": match.group(6), 934 "details": match.group(7), 935 } 936 data.append(entry) 937 # A reply that parses to nothing is nothing found, the same 938 # answer an empty reply gives, rather than an empty container 939 if not data: 940 return None 941 logger.debug( 942 "Found {} events of type {} in akips".format(len(data), event_type) 943 ) 944 return data 945 return None
Pull a list of events over a time period with optional filtering by device, child, attribute, and/or group membership. Defaults to all event types over the last hour. Review AKiPS documentation for details on event types and time filter syntax.
Supporting AKiPS command syntax:
mget event {all,critical,enum,threshold,uptime} time {time filter} [{parent regex} {child regex} {attribute regex}] [profile {profile name}] [any|all|not group {group name} ...]
Arguments:
- event_type (str): type of events to retrieve (default: 'all')
- period (str): time period to retrieve events from (default: 'last1h')
- device (str): device name or pattern to match (default: '*')
- child (str): child name or pattern to match (default: '*')
- attribute (str): attribute name or pattern to match (default: '*')
- group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any')
- groups (list): list of group names to filter by (if any)
Returns:
A list of event dictionaries, or None if no events found
Raises:
- AkipsError: if the AKiPS server returns an error
def
get_series( self, period: str = 'last1h', time_interval: int = 60, device: str = '*', attribute: str = '*', get_dict: bool = True, group_filter: str = 'any', groups: list[str] | None = None) -> list[dict[str, str]] | list[list[str]] | None:
949 def get_series( 950 self, 951 period: str = "last1h", 952 time_interval: int = 60, 953 device: str = "*", 954 attribute: str = "*", 955 get_dict: bool = True, 956 group_filter: str = "any", 957 groups: list[str] | None = None, 958 ) -> list[dict[str, str]] | list[list[str]] | None: 959 """ 960 Pull a series of counter values with average values over a time period with optional 961 filtering by device, attribute, and/or group membership. Defaults to all devices 962 and attributes over the last hour with 60 second intervals. Review AKiPS documentation 963 for details on time filter syntax. 964 965 Supporting AKiPS command syntax: 966 967 cseries [interval total|avg {secs}] time {time filter} 968 {type} {parent regex} {child regex} {attribute regex} 969 [profile {profile name}] [any|all|not group {group name} ...] 970 971 Args: 972 period (str): time period to retrieve series from (default: 'last1h') 973 time_interval (int): interval in seconds for series data points (default: 60) 974 device (str): device name or pattern to match (default: '*') 975 attribute (str): attribute name or pattern to match (default: '*') 976 get_dict (bool): return each row as a dictionary keyed by the 977 header row, rather than the CSV as sent (default: True). 978 The header carries one column heading per interval, which is 979 the time axis for the values under it. As dictionaries those 980 headings are the keys; as lists the header is the first entry, 981 so the list form has one row more than the dictionary form 982 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 983 groups (list): list of group names to filter by (if any) 984 Returns: 985 A list of series data rows (as dictionaries or lists), or None if no data found 986 Raises: 987 AkipsError: if the AKiPS server returns an error 988 """ 989 params = { 990 "cmds": f"cseries interval avg {time_interval} time {period} * {device} * {attribute}" 991 } 992 if groups: 993 group_list = " ".join(groups) 994 params["cmds"] += f" {group_filter} group {group_list}" 995 text = self._get(params=params) 996 if text: 997 # Rows as dictionaries keyed by the header row, or as the CSV as 998 # sent with that header kept as the first entry. The header is 999 # the time axis, one column heading per interval, so the list form 1000 # keeps it: without it the values underneath are readings with no 1001 # timestamps. The dictionary form does not need it separately 1002 # because those headings became its keys. 1003 csv_to_list = self._parse_csv(text, header=get_dict) 1004 data_rows = csv_to_list if get_dict else csv_to_list[1:] 1005 if not data_rows: 1006 # A header with nothing under it is an axis with no series on 1007 # it, which is nothing found rather than a result 1008 return None 1009 logger.debug("Found {} series entries".format(len(csv_to_list))) 1010 return csv_to_list 1011 return None
Pull a series of counter values with average values over a time period with optional filtering by device, attribute, and/or group membership. Defaults to all devices and attributes over the last hour with 60 second intervals. Review AKiPS documentation for details on time filter syntax.
Supporting AKiPS command syntax:
cseries [interval total|avg {secs}] time {time filter} {type} {parent regex} {child regex} {attribute regex} [profile {profile name}] [any|all|not group {group name} ...]
Arguments:
- period (str): time period to retrieve series from (default: 'last1h')
- time_interval (int): interval in seconds for series data points (default: 60)
- device (str): device name or pattern to match (default: '*')
- attribute (str): attribute name or pattern to match (default: '*')
- get_dict (bool): return each row as a dictionary keyed by the header row, rather than the CSV as sent (default: True). The header carries one column heading per interval, which is the time axis for the values under it. As dictionaries those headings are the keys; as lists the header is the first entry, so the list form has one row more than the dictionary form
- group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any')
- groups (list): list of group names to filter by (if any)
Returns:
A list of series data rows (as dictionaries or lists), or None if no data found
Raises:
- AkipsError: if the AKiPS server returns an error
def
get_latest_values( self, attribute: str, device: str = '*', child: str = '*', period: str = 'last1h', time_interval: int = 300, group_filter: str = 'any', groups: list[str] | None = None) -> dict[str, dict[str, dict[str, typing.Any]]] | None:
1013 def get_latest_values( 1014 self, 1015 attribute: str, 1016 device: str = "*", 1017 child: str = "*", 1018 period: str = "last1h", 1019 time_interval: int = 300, 1020 group_filter: str = "any", 1021 groups: list[str] | None = None, 1022 ) -> dict[str, dict[str, dict[str, Any]]] | None: 1023 """ 1024 Pull the most recent reading of a numeric attribute for each device. 1025 1026 Numeric attributes do not hold a reading in the config database that 1027 get_attributes() reads; that holds the counter or gauge definition, 1028 which is the same for every device. The readings live in the time 1029 series database, so this asks for a short series and keeps the last 1030 value in it. 1031 1032 The final interval of a series is usually still being filled and comes 1033 back empty, so the last column is not the answer; this returns the 1034 last column that has a value, along with when it was measured. Values 1035 are already scaled by AKiPS, so what comes back is in the attribute's 1036 real units. 1037 1038 Supporting AKiPS command syntax: 1039 1040 cseries [interval total|avg {secs}] time {time filter} 1041 {type} {parent regex} {child regex} {attribute regex} 1042 [profile {profile name}] [any|all|not group {group name} ...] 1043 1044 Args: 1045 attribute (str): the attribute to read 1046 device (str): device name or pattern to match (default: '*') 1047 child (str): child name or pattern to match (default: '*') 1048 period (str): how far back to look (default: 'last1h'). It only 1049 has to be long enough to contain one completed interval 1050 time_interval (int): seconds per interval (default: 300) 1051 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 1052 groups (list): list of group names to filter by (if any) 1053 Returns: 1054 A dictionary of device names to child names to the reading, each 1055 with 'value', 'time' and 'attribute'. A device with no reading in 1056 the period is present with a value of None rather than dropped. 1057 None if nothing matched at all 1058 Raises: 1059 AkipsError: if the AKiPS server returns an error 1060 """ 1061 params = { 1062 "cmds": f"cseries interval avg {time_interval} time {period} " 1063 f"* {device} {child} {attribute}" 1064 } 1065 if groups: 1066 # [any|all|not group {group name} ...] 1067 group_list = " ".join(groups) 1068 params["cmds"] += f" {group_filter} group {group_list}" 1069 text = self._get(params=params) 1070 if not text: 1071 return None 1072 1073 # The columns every cseries reply starts with, before the timestamps 1074 fixed_columns = ("parent", "child", "child description", "attribute") 1075 data: dict[str, dict[str, dict[str, Any]]] = {} 1076 unreadable = [] 1077 rows = cast(list[dict[str, str]], self._parse_csv(text, header=True)) 1078 for row in rows: 1079 parent = row.get("parent") 1080 child_name = row.get("child") 1081 if not parent or not child_name: 1082 continue 1083 # Everything after the fixed columns is a timestamped reading, in 1084 # order, because the reader keeps the header's column order 1085 readings = [ 1086 (column, value) 1087 for column, value in row.items() 1088 if column not in fixed_columns and value 1089 ] 1090 entry: dict[str, Any] = { 1091 "attribute": row.get("attribute", attribute), 1092 "value": None, 1093 "time": None, 1094 } 1095 if readings: 1096 column, value = readings[-1] 1097 try: 1098 entry["value"] = float(value) 1099 except ValueError: 1100 unreadable.append(f"{parent} {child_name} = {value}") 1101 continue 1102 try: 1103 entry["time"] = pytz.timezone(self.server_timezone).localize( 1104 datetime.strptime(column, "%Y-%m-%d %H:%M") 1105 ) 1106 except ValueError: 1107 # A column heading in a shape this does not recognize is 1108 # not worth losing the reading over 1109 entry["time"] = None 1110 data.setdefault(parent, {})[child_name] = entry 1111 1112 if unreadable: 1113 logger.warning( 1114 "Could not read {} of {} {} values from akips, those are " 1115 "missing from the result. First: {}".format( 1116 len(unreadable), 1117 len(unreadable) + len(rows), 1118 attribute, 1119 unreadable[0], 1120 ) 1121 ) 1122 if not data: 1123 return None 1124 logger.debug("Found readings for {} devices".format(len(data))) 1125 return data
Pull the most recent reading of a numeric attribute for each device.
Numeric attributes do not hold a reading in the config database that get_attributes() reads; that holds the counter or gauge definition, which is the same for every device. The readings live in the time series database, so this asks for a short series and keeps the last value in it.
The final interval of a series is usually still being filled and comes back empty, so the last column is not the answer; this returns the last column that has a value, along with when it was measured. Values are already scaled by AKiPS, so what comes back is in the attribute's real units.
Supporting AKiPS command syntax:
cseries [interval total|avg {secs}] time {time filter} {type} {parent regex} {child regex} {attribute regex} [profile {profile name}] [any|all|not group {group name} ...]
Arguments:
- attribute (str): the attribute to read
- device (str): device name or pattern to match (default: '*')
- child (str): child name or pattern to match (default: '*')
- period (str): how far back to look (default: 'last1h'). It only has to be long enough to contain one completed interval
- time_interval (int): seconds per interval (default: 300)
- group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any')
- groups (list): list of group names to filter by (if any)
Returns:
A dictionary of device names to child names to the reading, each with 'value', 'time' and 'attribute'. A device with no reading in the period is present with a value of None rather than dropped. None if nothing matched at all
Raises:
- AkipsError: if the AKiPS server returns an error
def
get_aggregate( self, period: str = 'last1h', device: str = '*', attribute: str = '*', operator: str = 'avg', time_interval: int = 300, group_filter: str = 'any', groups: list[str] | None = None, labeled: bool = False) -> list[str] | list[dict[str, typing.Any]] | None:
1127 def get_aggregate( 1128 self, 1129 period: str = "last1h", 1130 device: str = "*", 1131 attribute: str = "*", 1132 operator: str = "avg", 1133 time_interval: int = 300, 1134 group_filter: str = "any", 1135 groups: list[str] | None = None, 1136 labeled: bool = False, 1137 ) -> list[str] | list[dict[str, Any]] | None: 1138 """ 1139 Pull aggregate counter values over a period of time with optional filtering 1140 by device, attribute, and/or group membership. Defaults to all devices 1141 and attributes over the last hour with average aggregation every 300 seconds. Review 1142 AKiPS documentation for details on time filter syntax. 1143 1144 The aggregate collapses every matching device into a single series, so 1145 unlike get_series() the reply says nothing about what was measured. 1146 AKiPS sends no timestamps with it either, only the numbers, and there 1147 is one more of them than there are intervals: an hour at 300 seconds 1148 returns 13 values, not 12, the last of them landing on the end of the 1149 window. 1150 1151 Pass labeled=True to get a time against each value. That asks the 1152 server for the window with 'tf pairs' and spaces the values across it, 1153 which costs one extra request and is the only honest way to do it, 1154 since computing the axis here would be this module's clock rather than 1155 the server's. 1156 1157 An interval the server has no reading for comes back empty and is kept 1158 with a value of None, so the points stay in step with the axis. A 1159 calendar relative period returns a great many of those: 'last1d' is the 1160 whole of today, so at 300 seconds it is 288 intervals of which only the 1161 elapsed ones hold anything, and the rest are timestamped into the 1162 evening to come. Use 'last24h' for a rolling day with data throughout. 1163 1164 Supporting AKiPS command syntax: 1165 1166 aggregate [interval total|avg {secs}] time {time filter} 1167 {type} {parent regex} {child regex} {attribute regex} 1168 [profile {profile name}] [any|all|not group {group name} ...] 1169 1170 Args: 1171 period (str): time period to retrieve series from (default: 'last1h') 1172 device (str): device name or pattern to match (default: '*') 1173 attribute (str): attribute name or pattern to match (default: '*') 1174 operator (str): aggregation operator, 'avg' or 'total seconds' (default: 'avg') 1175 time_interval (int): seconds per aggregation point (default: 300). 1176 Named to match get_series() and get_latest_values(), which 1177 take the same thing 1178 group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any') 1179 groups (list): list of group names to filter by (if any) 1180 labeled (bool): put a time against each value (default: False). 1181 Adds a request, and needs a period covering one continuous 1182 range; a filter such as 'lastweek; mon to fri 8:00 to 17:00' 1183 is several disjoint ranges and cannot be one axis 1184 Returns: 1185 A list of aggregate values, one more than the number of intervals, 1186 or None if no data found. With labeled=True, a list of 1187 dictionaries with 'time' and 'value', where 'time' is timezone 1188 aware in the server's timezone and 'value' is a float, or None 1189 where the value was not a number 1190 Raises: 1191 ValueError: if labeled is asked for and the period does not 1192 describe one continuous range 1193 AkipsError: if the AKiPS server returns an error 1194 """ 1195 params = { 1196 "cmds": f"aggregate interval {operator} {time_interval} time {period} * {device} * {attribute}" 1197 } 1198 if groups: 1199 group_list = " ".join(groups) 1200 params["cmds"] += f" {group_filter} group {group_list}" 1201 text = self._get(params=params) 1202 if text: 1203 # One CSV row of values, followed by a blank line 1204 rows = cast(list[list[str]], self._parse_csv(text)) 1205 values = rows[0] if rows else [] 1206 if not values: 1207 return None 1208 logger.debug("Found {} aggregate values".format(len(values))) 1209 if not labeled: 1210 return values 1211 return self._label_aggregate(values, period, time_interval) 1212 return None
Pull aggregate counter values over a period of time with optional filtering by device, attribute, and/or group membership. Defaults to all devices and attributes over the last hour with average aggregation every 300 seconds. Review AKiPS documentation for details on time filter syntax.
The aggregate collapses every matching device into a single series, so unlike get_series() the reply says nothing about what was measured. AKiPS sends no timestamps with it either, only the numbers, and there is one more of them than there are intervals: an hour at 300 seconds returns 13 values, not 12, the last of them landing on the end of the window.
Pass labeled=True to get a time against each value. That asks the server for the window with 'tf pairs' and spaces the values across it, which costs one extra request and is the only honest way to do it, since computing the axis here would be this module's clock rather than the server's.
An interval the server has no reading for comes back empty and is kept with a value of None, so the points stay in step with the axis. A calendar relative period returns a great many of those: 'last1d' is the whole of today, so at 300 seconds it is 288 intervals of which only the elapsed ones hold anything, and the rest are timestamped into the evening to come. Use 'last24h' for a rolling day with data throughout.
Supporting AKiPS command syntax:
aggregate [interval total|avg {secs}] time {time filter} {type} {parent regex} {child regex} {attribute regex} [profile {profile name}] [any|all|not group {group name} ...]
Arguments:
- period (str): time period to retrieve series from (default: 'last1h')
- device (str): device name or pattern to match (default: '*')
- attribute (str): attribute name or pattern to match (default: '*')
- operator (str): aggregation operator, 'avg' or 'total seconds' (default: 'avg')
- time_interval (int): seconds per aggregation point (default: 300). Named to match get_series() and get_latest_values(), which take the same thing
- group_filter (str): 'any', 'all', or 'not' operators for group filtering (default: 'any')
- groups (list): list of group names to filter by (if any)
- labeled (bool): put a time against each value (default: False). Adds a request, and needs a period covering one continuous range; a filter such as 'lastweek; mon to fri 8:00 to 17:00' is several disjoint ranges and cannot be one axis
Returns:
A list of aggregate values, one more than the number of intervals, or None if no data found. With labeled=True, a list of dictionaries with 'time' and 'value', where 'time' is timezone aware in the server's timezone and 'value' is a float, or None where the value was not a number
Raises:
- ValueError: if labeled is asked for and the period does not describe one continuous range
- AkipsError: if the AKiPS server returns an error
def
cmd(self, cmd: str, output: str = 'raw') -> str | None:
1285 def cmd(self, cmd: str, output: str = "raw") -> str | None: 1286 """ 1287 Deprecated since 1.0.0, use call() instead, which reaches every API 1288 section and can parse the reply rather than only returning it raw. 1289 1290 Args: 1291 cmd (str): AKiPS command string to send 1292 output (str): desired output format, only 'raw' is supported 1293 Returns: 1294 The command output, or None if no output 1295 Raises: 1296 ValueError: if an invalid output format is provided 1297 AkipsError: if the AKiPS server returns an error 1298 """ 1299 warnings.warn( 1300 "cmd() is deprecated and will be removed in a future release, " 1301 "use call() instead", 1302 DeprecationWarning, 1303 stacklevel=2, 1304 ) 1305 if output != "raw": 1306 raise ValueError("Invalid output value provided to cmd.") 1307 return cast("str | None", self.call(command=cmd))
Deprecated since 1.0.0, use call() instead, which reaches every API section and can parse the reply rather than only returning it raw.
Arguments:
- cmd (str): AKiPS command string to send
- output (str): desired output format, only 'raw' is supported
Returns:
The command output, or None if no output
Raises:
- ValueError: if an invalid output format is provided
- AkipsError: if the AKiPS server returns an error
def
get_device_by_ip(self, ipaddr: str) -> str | None:
1312 def get_device_by_ip(self, ipaddr: str) -> str | None: 1313 """ 1314 Return the device name (primary key) for a device matching the given IP address. 1315 AKiPS records additional IP addresses when found on devices, so this function 1316 can be used to find the primary device name (primary key) from any known IP address. 1317 1318 A device is stored under one address, but it may answer on several. 1319 The case this exists for is a syslog message or SNMP trap arriving 1320 from an interface other than the one AKiPS knows the device by, where 1321 the source address matches no device at all. AKiPS keeps an internal 1322 address to device table, which the site script reads, so this maps any 1323 address the server has seen back to the device holding it. 1324 1325 That table is not the device's attributes, and searching the attributes 1326 is not a substitute. AKiPS keeps the addresses configured on a device 1327 in a CSV file rather than in its database, which is why an mget against 1328 attributes does not find them; the GUI shows the same file as its 1329 'Device to IP Mapping' report. Confirmed on a live server, where an 1330 address resolved here to a device whose entire attribute tree contained 1331 no mention of it. 1332 1333 This is the one read in this module that a read only deployment cannot 1334 perform. AKiPS exposes it as a site script rather than a database 1335 query, so it lives in api-script and needs rw_password even though it 1336 changes nothing. A client holding only ro_password raises 1337 AkipsCredentialError rather than returning None, so a caller building 1338 on it should not plan for a read only deployment. 1339 1340 **This call is sent as GET, not POST**, so its password travels in 1341 the query string. api-script does not answer a POST: the server sends 1342 no body and holds the connection open until the client gives up. See 1343 SECTION_METHODS, which is where to say so if a server of yours does 1344 take a POST on this section. 1345 1346 Supporting AKiPS site script function (which requires the api-rw user): 1347 1348 web_find_device_by_ip(ipaddr) 1349 1350 Args: 1351 ipaddr (str): IP address to search for 1352 Returns: 1353 the device name (str) if found, or None if no match is found 1354 Raises: 1355 AkipsError: if the AKiPS server returns an error 1356 """ 1357 params = {"function": "web_find_device_by_ip", "ipaddr": ipaddr} 1358 text = self._get(section="api-script", params=params) 1359 if not text: 1360 return None 1361 for line in text.split("\n"): 1362 match = re.match(r"IP Address (\S+) is configured on (\S+)", line) 1363 if match: 1364 address = match.group(1) 1365 device_name = match.group(2) 1366 logger.debug(f"Found {address} on device {device_name}") 1367 return device_name 1368 # The site script says so in as many words when it finds nothing, so a 1369 # reply that is neither that nor a match did not come from it. The 1370 # likeliest cause is the script not being installed, which would 1371 # otherwise read as 'no device has that address' and be believed. 1372 if "is not configured on any devices" not in text: 1373 logger.warning( 1374 "web_find_device_by_ip returned something unexpected; check " 1375 "that the site script is installed on this AKiPS server. " 1376 "Reply: {}".format(self._redact_text(text.strip()[:200])) 1377 ) 1378 return None
Return the device name (primary key) for a device matching the given IP address. AKiPS records additional IP addresses when found on devices, so this function can be used to find the primary device name (primary key) from any known IP address.
A device is stored under one address, but it may answer on several. The case this exists for is a syslog message or SNMP trap arriving from an interface other than the one AKiPS knows the device by, where the source address matches no device at all. AKiPS keeps an internal address to device table, which the site script reads, so this maps any address the server has seen back to the device holding it.
That table is not the device's attributes, and searching the attributes is not a substitute. AKiPS keeps the addresses configured on a device in a CSV file rather than in its database, which is why an mget against attributes does not find them; the GUI shows the same file as its 'Device to IP Mapping' report. Confirmed on a live server, where an address resolved here to a device whose entire attribute tree contained no mention of it.
This is the one read in this module that a read only deployment cannot perform. AKiPS exposes it as a site script rather than a database query, so it lives in api-script and needs rw_password even though it changes nothing. A client holding only ro_password raises AkipsCredentialError rather than returning None, so a caller building on it should not plan for a read only deployment.
This call is sent as GET, not POST, so its password travels in the query string. api-script does not answer a POST: the server sends no body and holds the connection open until the client gives up. See SECTION_METHODS, which is where to say so if a server of yours does take a POST on this section.
Supporting AKiPS site script function (which requires the api-rw user):
web_find_device_by_ip(ipaddr)
Arguments:
- ipaddr (str): IP address to search for
Returns:
the device name (str) if found, or None if no match is found
Raises:
- AkipsError: if the AKiPS server returns an error
def
set_group_membership(self, device: str, group: str, mode: str) -> None:
1380 def set_group_membership(self, device: str, group: str, mode: str) -> None: 1381 """ 1382 Update manual grouping rules for a device, including the special 'maintenance_mode' 1383 group. The web api script fails silently if the device or group does not exist. 1384 1385 **This call is sent as GET, not POST**, so its password travels in 1386 the query string. api-script does not answer a POST: the server sends 1387 no body and holds the connection open until the client gives up. See 1388 SECTION_METHODS, which is where to say so if a server of yours does 1389 take a POST on this section. 1390 1391 Supporting AKiPS site script function (which requires the api-rw user): 1392 1393 web_manual_grouping(type, group, mode, device) 1394 1395 Args: 1396 device (str): the AKiPS name of one device, exactly; this 1397 takes no pattern 1398 group (str): group name to update 1399 mode (str): 'assign' to add device to group, 'clear' to remove device from group 1400 Returns: 1401 None 1402 Raises: 1403 ValueError: if invalid parameters are provided 1404 AkipsError: if the AKiPS server returns an error 1405 """ 1406 if not device: 1407 raise ValueError( 1408 "a valid device name must be provided for manual grouping update" 1409 ) 1410 if not group: 1411 raise ValueError( 1412 "a valid group name must be provided for manual grouping update" 1413 ) 1414 if mode not in ("assign", "clear"): 1415 raise ValueError( 1416 "mode must be 'assign' or 'clear' for manual grouping update" 1417 ) 1418 params = { 1419 "function": "web_manual_grouping", 1420 "type": "device", 1421 "group": group, # group_name 1422 "mode": mode, # 'assign' or 'clear' for device memberships 1423 "device": device, # device_name 1424 } 1425 text = self._get(section="api-script", params=params) 1426 if text: 1427 logger.error("Web API request failed: {}".format(text)) 1428 raise AkipsError(message=text) 1429 return None
Update manual grouping rules for a device, including the special 'maintenance_mode' group. The web api script fails silently if the device or group does not exist.
This call is sent as GET, not POST, so its password travels in the query string. api-script does not answer a POST: the server sends no body and holds the connection open until the client gives up. See SECTION_METHODS, which is where to say so if a server of yours does take a POST on this section.
Supporting AKiPS site script function (which requires the api-rw user):
web_manual_grouping(type, group, mode, device)
Arguments:
- device (str): the AKiPS name of one device, exactly; this takes no pattern
- group (str): group name to update
- mode (str): 'assign' to add device to group, 'clear' to remove device from group
Returns:
None
Raises:
- ValueError: if invalid parameters are provided
- AkipsError: if the AKiPS server returns an error
SCRIPT_TIMEOUT =
300
Seconds a site script that does work is given, in place of the client's timeout.
Site scripts divide into two kinds. Most answer a question and return at once — get_device_by_ip() and set_group_membership() are ordinary requests and keep the client's timeout, so a hung one fails as promptly as any other call. A few go away and do something: deleting a device today, and discovery, rewalk and rename if those are ever wrapped. Those are what this is for.
It is not per method on purpose. Every long running script wants the same thing — more room than a read gets — and a constant for each would be a new name to learn for every script added. A method needing something different takes a timeout argument instead.
Why generous: a timeout part way through work that changes the server leaves the worst of the three outcomes, where the caller cannot tell whether it happened, since nothing can confirm an outcome when the call itself raises. Waiting longer costs only waiting.
How long any of these really take is not known. The one timing on record, a delete just past 30 seconds against a 30 second timeout, was taken while api-script still hung on every POST, so it measures the client giving up rather than the work — see SECTION_METHODS.
A client configured with a longer timeout than this keeps it; this is a floor, not a ceiling.
def
delete_device(self, device: str, timeout: int | None = None) -> bool:
1460 def delete_device(self, device: str, timeout: int | None = None) -> bool: 1461 """ 1462 Delete one device from AKiPS. 1463 1464 **This cannot be undone.** Whether the samples, events and 1465 availability held against the device go with it is a property of 1466 AKiPS's own config_delete_device built in, which the site script calls 1467 and this module cannot see into, so treat the whole record as lost 1468 until AKiPS says otherwise. There is no merge: where the same box is 1469 registered twice under two names, copy whatever the surviving record 1470 should keep before deleting the other one, because nothing moves 1471 across on its own. 1472 1473 **This call is sent as GET, not POST**, so its password travels in 1474 the query string. api-script does not answer a POST: the server sends 1475 no body and holds the connection open until the client gives up. See 1476 SECTION_METHODS, which is where to say so if a server of yours does 1477 take a POST on this section. 1478 1479 Supporting AKiPS site script function (which requires the api-rw user): 1480 1481 web_delete_device(device_names) 1482 1483 AKiPS publishes that script and does not install it by default; see 1484 akips_setup/README.md. It prints nothing whether it worked or not, so 1485 this method confirms the outcome rather than trusting the silence. It 1486 checks the device is there first, which is how a name that never 1487 existed is told apart from one that was removed, and checks it is gone 1488 afterwards, which is how a script that quietly did nothing is caught. 1489 That costs two extra requests, which is the right trade for an 1490 operation with no undo. 1491 1492 **An exception does not mean nothing happened.** The confirmation 1493 below cannot run when the call itself fails, and AKiPS finishes the 1494 work whether or not the client is still listening: a delete that ran 1495 just past a 30 second timeout removed the device and raised anyway, 1496 so the caller recorded a failure against a device already gone. On any exception, ask AKiPS again rather than 1497 recording a failure. Gone, still there, and could not tell are three 1498 different outcomes and only the first two are knowable from here. 1499 1500 Args: 1501 device (str): the AKiPS name of one device, exactly. This takes 1502 no pattern and no list. A name holding a comma or an asterisk 1503 is refused: the site script splits its argument on commas, so 1504 such a name would delete more than was asked for, and a 1505 partial or oversized delete cannot be walked back. 1506 timeout (int): seconds to wait for the delete itself. Defaults 1507 to SCRIPT_TIMEOUT, or the client's timeout if that is longer, 1508 because a timeout during a destructive call leaves an outcome 1509 nobody can read. The two lookups either side are ordinary 1510 reads and use the client's timeout. 1511 Returns: 1512 True if the device was deleted, False if there was no such device. 1513 The two are distinguishable on purpose, so a caller does not 1514 report success for a name that was never there. 1515 Raises: 1516 ValueError: if device is empty, is a pattern, or could name more 1517 than one device 1518 AkipsCredentialError: if no rw_password was given to AKIPS() 1519 AkipsError: if AKiPS returns an error, or if the device is still 1520 present afterwards 1521 """ 1522 if not device: 1523 raise ValueError("a device name must be provided to delete a device") 1524 if device.startswith("/") and device.endswith("/") and len(device) > 1: 1525 raise ValueError( 1526 "delete_device takes one device name, not a pattern. Got " 1527 "{!r}".format(device) 1528 ) 1529 for char in (",", "*"): 1530 if char in device: 1531 # web_delete_device does cgi_param("device_names") in scalar 1532 # context and splits on commas itself, so a comma here is not 1533 # an odd name but a second device. Refused rather than 1534 # escaped, because there is no undo to fall back on. 1535 raise ValueError( 1536 "refusing to delete {!r}: a name containing {!r} can match " 1537 "more than one device, and this cannot be undone".format( 1538 device, char 1539 ) 1540 ) 1541 1542 # Checked before anything is looked up, so a client with no rw 1543 # password fails on the credential rather than after spending a 1544 # request on a delete it could never have made. 1545 self._credentials_for("api-script") 1546 1547 if self.get_device(device) is None: 1548 logger.info("No AKiPS device named {!r}, nothing to delete".format(device)) 1549 return False 1550 1551 params = { 1552 "function": "web_delete_device", 1553 "device_names": device, # one name; the script splits on commas 1554 } 1555 if timeout is None: 1556 # A floor rather than a replacement: a client deliberately given 1557 # longer than this keeps it. 1558 timeout = max(self.timeout, self.SCRIPT_TIMEOUT) 1559 text = self._get(section="api-script", params=params, timeout=timeout) 1560 if text: 1561 logger.error("Web API request failed: {}".format(text)) 1562 raise AkipsError(message=text) 1563 1564 if self.get_device(device) is not None: 1565 raise AkipsError( 1566 message=( 1567 "AKiPS still holds a device named {!r} after " 1568 "web_delete_device returned nothing. Check that the site " 1569 "script is installed and that api-rw is allowed to run " 1570 "it".format(device) 1571 ) 1572 ) 1573 logger.info("Deleted AKiPS device {!r} and its history".format(device)) 1574 return True
Delete one device from AKiPS.
This cannot be undone. Whether the samples, events and availability held against the device go with it is a property of AKiPS's own config_delete_device built in, which the site script calls and this module cannot see into, so treat the whole record as lost until AKiPS says otherwise. There is no merge: where the same box is registered twice under two names, copy whatever the surviving record should keep before deleting the other one, because nothing moves across on its own.
This call is sent as GET, not POST, so its password travels in the query string. api-script does not answer a POST: the server sends no body and holds the connection open until the client gives up. See SECTION_METHODS, which is where to say so if a server of yours does take a POST on this section.
Supporting AKiPS site script function (which requires the api-rw user):
web_delete_device(device_names)
AKiPS publishes that script and does not install it by default; see akips_setup/README.md. It prints nothing whether it worked or not, so this method confirms the outcome rather than trusting the silence. It checks the device is there first, which is how a name that never existed is told apart from one that was removed, and checks it is gone afterwards, which is how a script that quietly did nothing is caught. That costs two extra requests, which is the right trade for an operation with no undo.
An exception does not mean nothing happened. The confirmation below cannot run when the call itself fails, and AKiPS finishes the work whether or not the client is still listening: a delete that ran just past a 30 second timeout removed the device and raised anyway, so the caller recorded a failure against a device already gone. On any exception, ask AKiPS again rather than recording a failure. Gone, still there, and could not tell are three different outcomes and only the first two are knowable from here.
Arguments:
- device (str): the AKiPS name of one device, exactly. This takes no pattern and no list. A name holding a comma or an asterisk is refused: the site script splits its argument on commas, so such a name would delete more than was asked for, and a partial or oversized delete cannot be walked back.
- timeout (int): seconds to wait for the delete itself. Defaults to SCRIPT_TIMEOUT, or the client's timeout if that is longer, because a timeout during a destructive call leaves an outcome nobody can read. The two lookups either side are ordinary reads and use the client's timeout.
Returns:
True if the device was deleted, False if there was no such device. The two are distinguishable on purpose, so a caller does not report success for a name that was never there.
Raises:
- ValueError: if device is empty, is a pattern, or could name more than one device
- AkipsCredentialError: if no rw_password was given to AKIPS()
- AkipsError: if AKiPS returns an error, or if the device is still present afterwards
def
get_msg( self, period: str = 'last1h', addr: str | None = None, msg_type: str | None = None, device: str | None = None, regex: str | None = None, limit: int | None = None) -> list[dict[str, str]] | None:
1589 def get_msg( 1590 self, 1591 period: str = "last1h", 1592 addr: str | None = None, 1593 msg_type: str | None = None, 1594 device: str | None = None, 1595 regex: str | None = None, 1596 limit: int | None = None, 1597 ) -> list[dict[str, str]] | None: 1598 """ 1599 Retrieve syslog or trap messages from the AKiPS api-msg database. The api-msg 1600 access requires username to be 'api-ro'. 1601 1602 Supporting AKiPS web API syntax: 1603 1604 https://{server}/api-msg?password={pw};time={time filter}; 1605 [addr={ip filter}];[type=syslog|trap];[device={name}|{regex}]; 1606 [regex={regex filter}];[limit={qty messages}] 1607 1608 This is the highest volume call here, and worth filtering. Measured on 1609 a 17,000 device fleet, an unfiltered 'last1h' returned 472,014 messages 1610 in 5.5 seconds; the same hour asking only for traps returned 5,160 in 1611 0.8 seconds. Syslog is the bulk of it, and a single appliance can be a 1612 large share of that on its own. See get_traps() and get_syslog(). 1613 1614 Args: 1615 period (str): Required, time period to retrieve messages from 1616 (default: 'last1h'). 'lastNm' and 'lastNh' are rolling 1617 windows of the length they name, measured back from the moment 1618 of the call. 'lastNd' is calendar relative, meaning N-1 whole 1619 days plus today so far, so 'last1d' is today rather than 24 1620 hours; use 'last24h' for a rolling day 1621 addr (str): IP address to filter messages by (default: None) 1622 msg_type (str): message type, 'syslog' or 'trap', or None for both 1623 (default: None). See get_syslog() and get_traps(), which name 1624 the type rather than asking a caller to spell it 1625 device (str): device name to filter messages by (default: None) 1626 regex (str): regex pattern to filter message content by (default: None) 1627 limit (int): maximum number of messages to return (default: None). 1628 AKiPS fills this from the start of the window, so by default it 1629 returns the oldest matching messages rather than the newest, 1630 and there is no ordering parameter on the request. AKiPS 25.6 1631 added a reverse sort option under Miscellaneous Settings, which 1632 is server wide rather than per call; whether it reaches this 1633 section has not been tested here. For recent activity narrow 1634 'period' instead: 'last15m' with no limit costs far less than 1635 an hour of messages thrown away after the fact 1636 Returns: 1637 A list of dictionaries, each with 'time', 'type', 'ip_ver', 1638 'ip_addr' and 'message', or None if no data found. 'ip_addr' is 1639 where the message came from, which need not be the address AKiPS 1640 holds for the device: a device with several interfaces can send 1641 from any of them. get_device_by_ip() resolves one to a device 1642 Raises: 1643 ValueError: if msg_type is not 'syslog', 'trap' or None 1644 AkipsError: if the AKiPS server returns an error 1645 """ 1646 1647 # Checked rather than quietly ignored. An unrecognized type used to be 1648 # dropped, so a caller asking for 'traps' or 'Syslog' was sent no type 1649 # at all and got both back believing it had filtered to one. 1650 if msg_type is not None and msg_type not in self.MSG_TYPES: 1651 raise ValueError( 1652 "Invalid msg_type provided to get_msg, expected one of {}, " 1653 "or None for both".format(", ".join(self.MSG_TYPES)) 1654 ) 1655 params = {"time": period} 1656 if msg_type is not None: 1657 params["type"] = msg_type 1658 if addr: 1659 params["addr"] = addr 1660 if device: 1661 params["device"] = device 1662 if regex: 1663 params["regex"] = regex 1664 if limit: 1665 params["limit"] = str(limit) 1666 text = self._get(section="api-msg", params=params) 1667 if text: 1668 # Each syslog or trap message contains: 1669 # header line: {system timestamp} {type} {IP version} {IP address} 1670 # message line(s): {message text} 1671 # blank terminating line 1672 # 1673 # Records are split on that blank line rather than by recognizing 1674 # each header, because a body line can look exactly like a header 1675 # and would otherwise start a new record in the middle of a 1676 # message, turning one message into two with empty bodies. 1677 data = [] 1678 unparsed = 0 1679 for record in re.split(r"\n\s*\n", text): 1680 lines = [line for line in record.split("\n") if line.strip()] 1681 if not lines: 1682 continue 1683 header = re.match( 1684 r"^(?P<time>\S+)\s(?P<type>\S+)\s(?P<ip_ver>[46])\s(?P<ip_addr>\S+)$", 1685 lines[0], 1686 ) 1687 if not header: 1688 unparsed += 1 1689 continue 1690 data.append( 1691 { 1692 "time": header.group("time"), 1693 "type": header.group("type"), 1694 "ip_ver": header.group("ip_ver"), 1695 "ip_addr": header.group("ip_addr"), 1696 # Everything after the header is the message, whatever 1697 # any of those lines happen to look like 1698 "message": "\n".join(lines[1:]), 1699 } 1700 ) 1701 if unparsed: 1702 logger.warning( 1703 "Could not parse {} of {} message records from akips, " 1704 "those messages are missing from the result".format( 1705 unparsed, unparsed + len(data) 1706 ) 1707 ) 1708 # A reply that parses to nothing is nothing found, the same 1709 # answer an empty reply gives, rather than an empty container 1710 if not data: 1711 return None 1712 logger.debug("Found {} messages in akips".format(len(data))) 1713 return data 1714 return None
Retrieve syslog or trap messages from the AKiPS api-msg database. The api-msg access requires username to be 'api-ro'.
Supporting AKiPS web API syntax:
https://{server}/api-msg?password={pw};time={time filter}; [addr={ip filter}];[type=syslog|trap];[device={name}|{regex}]; [regex={regex filter}];[limit={qty messages}]
This is the highest volume call here, and worth filtering. Measured on a 17,000 device fleet, an unfiltered 'last1h' returned 472,014 messages in 5.5 seconds; the same hour asking only for traps returned 5,160 in 0.8 seconds. Syslog is the bulk of it, and a single appliance can be a large share of that on its own. See get_traps() and get_syslog().
Arguments:
- period (str): Required, time period to retrieve messages from (default: 'last1h'). 'lastNm' and 'lastNh' are rolling windows of the length they name, measured back from the moment of the call. 'lastNd' is calendar relative, meaning N-1 whole days plus today so far, so 'last1d' is today rather than 24 hours; use 'last24h' for a rolling day
- addr (str): IP address to filter messages by (default: None)
- msg_type (str): message type, 'syslog' or 'trap', or None for both (default: None). See get_syslog() and get_traps(), which name the type rather than asking a caller to spell it
- device (str): device name to filter messages by (default: None)
- regex (str): regex pattern to filter message content by (default: None)
- limit (int): maximum number of messages to return (default: None). AKiPS fills this from the start of the window, so by default it returns the oldest matching messages rather than the newest, and there is no ordering parameter on the request. AKiPS 25.6 added a reverse sort option under Miscellaneous Settings, which is server wide rather than per call; whether it reaches this section has not been tested here. For recent activity narrow 'period' instead: 'last15m' with no limit costs far less than an hour of messages thrown away after the fact
Returns:
A list of dictionaries, each with 'time', 'type', 'ip_ver', 'ip_addr' and 'message', or None if no data found. 'ip_addr' is where the message came from, which need not be the address AKiPS holds for the device: a device with several interfaces can send from any of them. get_device_by_ip() resolves one to a device
Raises:
- ValueError: if msg_type is not 'syslog', 'trap' or None
- AkipsError: if the AKiPS server returns an error
def
get_syslog( self, period: str = 'last1h', addr: str | None = None, device: str | None = None, regex: str | None = None, limit: int | None = None) -> list[dict[str, str]] | None:
1716 def get_syslog( 1717 self, 1718 period: str = "last1h", 1719 addr: str | None = None, 1720 device: str | None = None, 1721 regex: str | None = None, 1722 limit: int | None = None, 1723 ) -> list[dict[str, str]] | None: 1724 """ 1725 Retrieve syslog messages, leaving traps out. 1726 1727 The same as get_msg(msg_type='syslog') with every other filter 1728 forwarded, named so the type does not have to be spelled correctly to 1729 take effect. 1730 1731 Syslog is the high volume half of api-msg: an unfiltered hour was 1732 465,936 messages on a 17,000 device fleet, one appliance accounting 1733 for a large share of it. Pass a shorter period, a device or a regex 1734 unless the whole of it is wanted. 1735 1736 Args: 1737 period (str): time period to retrieve messages from 1738 (default: 'last1h'). 'lastNm' and 'lastNh' are rolling 1739 windows of the length they name; 'lastNd' is calendar 1740 relative, so 'last1d' is today rather than 24 hours 1741 addr (str): IP address to filter messages by (default: None) 1742 device (str): device name to filter messages by (default: None) 1743 regex (str): regex pattern to filter message content by 1744 (default: None) 1745 limit (int): maximum number of messages to return (default: None). 1746 This returns the oldest matching messages by default, not the 1747 newest; narrow 'period' for recent activity. See get_msg() 1748 for the server setting that may reverse it 1749 Returns: 1750 A list of dictionaries, each with 'time', 'type', 'ip_ver', 1751 'ip_addr' and 'message', or None if no data found. 'ip_addr' is 1752 where the message came from, which need not be the address AKiPS 1753 holds for the device: a device with several interfaces can send 1754 from any of them. get_device_by_ip() resolves one to a device 1755 Raises: 1756 AkipsError: if the AKiPS server returns an error 1757 """ 1758 return self.get_msg( 1759 period=period, 1760 msg_type="syslog", 1761 addr=addr, 1762 device=device, 1763 regex=regex, 1764 limit=limit, 1765 )
Retrieve syslog messages, leaving traps out.
The same as get_msg(msg_type='syslog') with every other filter forwarded, named so the type does not have to be spelled correctly to take effect.
Syslog is the high volume half of api-msg: an unfiltered hour was 465,936 messages on a 17,000 device fleet, one appliance accounting for a large share of it. Pass a shorter period, a device or a regex unless the whole of it is wanted.
Arguments:
- period (str): time period to retrieve messages from (default: 'last1h'). 'lastNm' and 'lastNh' are rolling windows of the length they name; 'lastNd' is calendar relative, so 'last1d' is today rather than 24 hours
- addr (str): IP address to filter messages by (default: None)
- device (str): device name to filter messages by (default: None)
- regex (str): regex pattern to filter message content by (default: None)
- limit (int): maximum number of messages to return (default: None). This returns the oldest matching messages by default, not the newest; narrow 'period' for recent activity. See get_msg() for the server setting that may reverse it
Returns:
A list of dictionaries, each with 'time', 'type', 'ip_ver', 'ip_addr' and 'message', or None if no data found. 'ip_addr' is where the message came from, which need not be the address AKiPS holds for the device: a device with several interfaces can send from any of them. get_device_by_ip() resolves one to a device
Raises:
- AkipsError: if the AKiPS server returns an error
def
get_traps( self, period: str = 'last1h', addr: str | None = None, device: str | None = None, regex: str | None = None, limit: int | None = None) -> list[dict[str, str]] | None:
1767 def get_traps( 1768 self, 1769 period: str = "last1h", 1770 addr: str | None = None, 1771 device: str | None = None, 1772 regex: str | None = None, 1773 limit: int | None = None, 1774 ) -> list[dict[str, str]] | None: 1775 """ 1776 Retrieve SNMP traps, leaving syslog out. 1777 1778 The same as get_msg(msg_type='trap') with every other filter 1779 forwarded, named so the type does not have to be spelled correctly to 1780 take effect. 1781 1782 Asking for traps is what makes this call cheap enough to poll: on a 1783 17,000 device fleet an hour of traps was 5,160 messages against 1784 472,014 for an unfiltered hour. 1785 1786 The body of a trap is a varbind list, one per line, as 1787 '{module} {attribute} {instance} {type} {value}'. It is returned as 1788 the raw 'message' text; this does not split it up. 1789 1790 Args: 1791 period (str): time period to retrieve messages from 1792 (default: 'last1h'). 'lastNm' and 'lastNh' are rolling 1793 windows of the length they name; 'lastNd' is calendar 1794 relative, so 'last1d' is today rather than 24 hours 1795 addr (str): IP address to filter messages by (default: None) 1796 device (str): device name to filter messages by (default: None) 1797 regex (str): regex pattern to filter message content by 1798 (default: None) 1799 limit (int): maximum number of messages to return (default: None). 1800 This returns the oldest matching messages by default, not the 1801 newest; narrow 'period' for recent activity. See get_msg() 1802 for the server setting that may reverse it 1803 Returns: 1804 A list of dictionaries, each with 'time', 'type', 'ip_ver', 1805 'ip_addr' and 'message', or None if no data found. 'ip_addr' is 1806 where the message came from, which need not be the address AKiPS 1807 holds for the device: a device with several interfaces can send 1808 from any of them. get_device_by_ip() resolves one to a device 1809 Raises: 1810 AkipsError: if the AKiPS server returns an error 1811 """ 1812 return self.get_msg( 1813 period=period, 1814 msg_type="trap", 1815 addr=addr, 1816 device=device, 1817 regex=regex, 1818 limit=limit, 1819 )
Retrieve SNMP traps, leaving syslog out.
The same as get_msg(msg_type='trap') with every other filter forwarded, named so the type does not have to be spelled correctly to take effect.
Asking for traps is what makes this call cheap enough to poll: on a 17,000 device fleet an hour of traps was 5,160 messages against 472,014 for an unfiltered hour.
The body of a trap is a varbind list, one per line, as '{module} {attribute} {instance} {type} {value}'. It is returned as the raw 'message' text; this does not split it up.
Arguments:
- period (str): time period to retrieve messages from (default: 'last1h'). 'lastNm' and 'lastNh' are rolling windows of the length they name; 'lastNd' is calendar relative, so 'last1d' is today rather than 24 hours
- addr (str): IP address to filter messages by (default: None)
- device (str): device name to filter messages by (default: None)
- regex (str): regex pattern to filter message content by (default: None)
- limit (int): maximum number of messages to return (default: None). This returns the oldest matching messages by default, not the newest; narrow 'period' for recent activity. See get_msg() for the server setting that may reverse it
Returns:
A list of dictionaries, each with 'time', 'type', 'ip_ver', 'ip_addr' and 'message', or None if no data found. 'ip_addr' is where the message came from, which need not be the address AKiPS holds for the device: a device with several interfaces can send from any of them. get_device_by_ip() resolves one to a device
Raises:
- AkipsError: if the AKiPS server returns an error
AVAILABILITY_PERIOD =
'last24h'
The period the availability methods use by default, a rolling 24 hours rather than 'last1d', which AKiPS reads as today so far.
def
get_group_availability( self, period: str = 'last24h', report: str = 'ping4', group: str | None = None) -> list[dict[str, str]] | None:
1837 def get_group_availability( 1838 self, 1839 period: str = AVAILABILITY_PERIOD, 1840 report: str = "ping4", 1841 group: str | None = None, 1842 ) -> list[dict[str, str]] | None: 1843 """ 1844 Retrieve availability statistics for a group of devices over a time period. 1845 1846 # output format: {child},{attr},{group name},{total time},{match time},{group target},{tf}[;{group tf}] 1847 # example: nm-availability mode group time last1w report ping4 1848 1849 ping4,PING.icmpState,1-Building-4,11688115,11687711,9990,last1w 1850 ping4,PING.icmpState,1-Fraser,8213270,8213190,9990,last1w 1851 ping4,PING.icmpState,1-Building-16,44541195,44540002,9990,last1w 1852 ping4,PING.icmpState,Accedian,1766635,1766635,9890,last1w;mon to sat 6:00 to 20:00 1853 ping4,PING.icmpState,Aerohive,589475,589475,9999,last1w;mon to fri 7:00 to 19:00; sat 8:00 to 18:00 1854 1855 Args: 1856 period (str): time filter, refer to the AKiPS programming guide 1857 (default: 'last24h'). 'lastNd' is calendar relative, meaning 1858 N-1 whole days plus today so far, so 'last1d' is today rather 1859 than 24 hours and shrinks to minutes just after midnight. 1860 'lastNh' and 'lastNm' are rolling windows of the length they 1861 name. 'total time' in the reply is the window measured 1862 report (str): 'ping4', 'ping6', 'snmp' or 'ifstatus', in any 1863 combination, comma separated (default: 'ping4') 1864 group (str): group name to filter by, or every group 1865 Returns: 1866 A list of dictionaries, one per group, or None if nothing matched 1867 Raises: 1868 AkipsError: if the AKiPS server returns an error 1869 """ 1870 params = { 1871 "maintenance": "off", # 'on' or 'off', show/hide maintenance mode devices 1872 "mode": "group", # 'group', 'device' or 'events' 1873 "time": period, # time filter, refer to programming guide 1874 "report": report, # 'ping4', 'ping6', 'snmp', 'ifstatus'. Any combination, comma separated, 1875 # "entity": device, # {device} [{child}] to filter by device or child 1876 "group": group, # {group name} to filter by group 1877 # "profile": "" # {profile name} to filter by profile 1878 } 1879 text = self._get(section="api-availability", params=params) 1880 if text: 1881 # This endpoint sends no header row, so the column names come from 1882 # here rather than from the reply 1883 column_headers = [ 1884 "child", 1885 "attr", 1886 "group name", 1887 "total time", 1888 "match time", 1889 "group target", 1890 "tf", 1891 ] 1892 csv_to_list = self._parse_csv(text, fieldnames=column_headers) 1893 logger.debug("Found {} entries".format(len(csv_to_list))) 1894 return cast("list[dict[str, str]]", csv_to_list) 1895 return None
Retrieve availability statistics for a group of devices over a time period.
output format: {child},{attr},{group name},{total time},{match time},{group target},{tf}[;{group tf}]
example: nm-availability mode group time last1w report ping4
ping4,PING.icmpState,1-Building-4,11688115,11687711,9990,last1w ping4,PING.icmpState,1-Fraser,8213270,8213190,9990,last1w ping4,PING.icmpState,1-Building-16,44541195,44540002,9990,last1w ping4,PING.icmpState,Accedian,1766635,1766635,9890,last1w;mon to sat 6:00 to 20:00 ping4,PING.icmpState,Aerohive,589475,589475,9999,last1w;mon to fri 7:00 to 19:00; sat 8:00 to 18:00
Arguments:
- period (str): time filter, refer to the AKiPS programming guide (default: 'last24h'). 'lastNd' is calendar relative, meaning N-1 whole days plus today so far, so 'last1d' is today rather than 24 hours and shrinks to minutes just after midnight. 'lastNh' and 'lastNm' are rolling windows of the length they name. 'total time' in the reply is the window measured
- report (str): 'ping4', 'ping6', 'snmp' or 'ifstatus', in any combination, comma separated (default: 'ping4')
- group (str): group name to filter by, or every group
Returns:
A list of dictionaries, one per group, or None if nothing matched
Raises:
- AkipsError: if the AKiPS server returns an error
def
get_device_availability( self, period: str = 'last24h', report: str = 'ping4', device: str | None = None, group: str | None = None) -> list[dict[str, str]] | None:
1897 def get_device_availability( 1898 self, 1899 period: str = AVAILABILITY_PERIOD, 1900 report: str = "ping4", 1901 device: str | None = None, 1902 group: str | None = None, 1903 ) -> list[dict[str, str]] | None: 1904 """ 1905 Retrieve availability statistics per device over a time period. 1906 1907 Where group mode summarises a whole group, this reports each device 1908 and child separately, so a device checked by both ping and SNMP 1909 appears on two rows. 1910 1911 A device or a group is required. Unlike group mode, device mode 1912 answers an unscoped call with an empty body rather than an error, 1913 which would reach the caller as None and read as 'nothing to report'. 1914 1915 # output format: {parent},{child},{attr},{total time},{match time},{group target} 1916 # example: nm-availability mode device time last1w report snmp,ping4 group Accedian 1917 1918 accedian-131-2-7,ping4,PING.icmpState,136020,136020,9890 1919 accedian-131-2-7,sys,SNMP.snmpState,136020,136020,9890 1920 accedian-131-2-8,ping4,PING.icmpState,136020,136020,9890 1921 accedian-131-2-8,sys,SNMP.snmpState,136020,136020,9890 1922 1923 'group target' is the availability AKiPS is configured to expect, in 1924 basis points, so 9890 is 98.90% and 10000 is 100.00%. It is set per 1925 group, so a caller can report against the target already agreed on 1926 the server rather than inventing a threshold of its own. 1927 1928 Args: 1929 period (str): time filter, refer to the AKiPS programming guide 1930 (default: 'last24h'). 'lastNd' is calendar relative, meaning 1931 N-1 whole days plus today so far, so 'last1d' is today rather 1932 than 24 hours and shrinks to minutes just after midnight. 1933 'lastNh' and 'lastNm' are rolling windows of the length they 1934 name. 'total time' in the reply is the window measured 1935 report (str): 'ping4', 'ping6', 'snmp' or 'ifstatus', in any 1936 combination, comma separated (default: 'ping4') 1937 device (str): device to filter by, as '{device}' or 1938 '{device} {child}'. This is the device's AKiPS name 1939 exactly, its one primary key, which is either its sysName or 1940 its IP address depending on how the server names devices, and 1941 takes no pattern; get_device_by_ip() resolves an address to it 1942 group (str): group name to filter by 1943 Returns: 1944 A list of dictionaries, one per device and child, or None if 1945 nothing matched 1946 Raises: 1947 ValueError: if neither device nor group is given 1948 AkipsError: if the AKiPS server returns an error 1949 """ 1950 # Checked before the request, so a call that could only ever come back 1951 # empty fails as the mistake it is rather than as good news 1952 if device is None and group is None: 1953 raise ValueError( 1954 "get_device_availability needs a device or a group to scope it, " 1955 "an unscoped call returns nothing at all" 1956 ) 1957 params = { 1958 "maintenance": "off", # 'on' or 'off', show/hide maintenance mode devices 1959 "mode": "device", # 'group', 'device' or 'events' 1960 "time": period, # time filter, refer to programming guide 1961 "report": report, # 'ping4', 'ping6', 'snmp', 'ifstatus'. Any combination, comma separated, 1962 # 'entity' is not in the nm-availability syntax the AKiPS API 1963 # guide publishes, which lists only mode, time, report, group and 1964 # profile. It works, and is how device and event mode are scoped 1965 # here, but being undocumented it is the parameter most likely to 1966 # change under us in a future AKiPS release. 1967 "entity": device, # {device} [{child}] to filter by device or child 1968 "group": group, # {group name} to filter by group 1969 } 1970 text = self._get(section="api-availability", params=params) 1971 if text: 1972 # This endpoint sends no header row, so the column names come from 1973 # here rather than from the reply. They are not group mode's 1974 # columns; each mode of nm-availability returns its own. 1975 column_headers = [ 1976 "parent", 1977 "child", 1978 "attr", 1979 "total time", 1980 "match time", 1981 "group target", 1982 ] 1983 csv_to_list = self._parse_csv(text, fieldnames=column_headers) 1984 logger.debug("Found {} entries".format(len(csv_to_list))) 1985 return cast("list[dict[str, str]]", csv_to_list) 1986 return None
Retrieve availability statistics per device over a time period.
Where group mode summarises a whole group, this reports each device and child separately, so a device checked by both ping and SNMP appears on two rows.
A device or a group is required. Unlike group mode, device mode answers an unscoped call with an empty body rather than an error, which would reach the caller as None and read as 'nothing to report'.
output format: {parent},{child},{attr},{total time},{match time},{group target}
example: nm-availability mode device time last1w report snmp,ping4 group Accedian
accedian-131-2-7,ping4,PING.icmpState,136020,136020,9890 accedian-131-2-7,sys,SNMP.snmpState,136020,136020,9890 accedian-131-2-8,ping4,PING.icmpState,136020,136020,9890 accedian-131-2-8,sys,SNMP.snmpState,136020,136020,9890
'group target' is the availability AKiPS is configured to expect, in basis points, so 9890 is 98.90% and 10000 is 100.00%. It is set per group, so a caller can report against the target already agreed on the server rather than inventing a threshold of its own.
Arguments:
- period (str): time filter, refer to the AKiPS programming guide (default: 'last24h'). 'lastNd' is calendar relative, meaning N-1 whole days plus today so far, so 'last1d' is today rather than 24 hours and shrinks to minutes just after midnight. 'lastNh' and 'lastNm' are rolling windows of the length they name. 'total time' in the reply is the window measured
- report (str): 'ping4', 'ping6', 'snmp' or 'ifstatus', in any combination, comma separated (default: 'ping4')
- device (str): device to filter by, as '{device}' or '{device} {child}'. This is the device's AKiPS name exactly, its one primary key, which is either its sysName or its IP address depending on how the server names devices, and takes no pattern; get_device_by_ip() resolves an address to it
- group (str): group name to filter by
Returns:
A list of dictionaries, one per device and child, or None if nothing matched
Raises:
- ValueError: if neither device nor group is given
- AkipsError: if the AKiPS server returns an error
def
get_event_availability( self, period: str = 'last24h', report: str = 'ping4', device: str | None = None, group: str | None = None) -> list[dict[str, str]] | None:
1988 def get_event_availability( 1989 self, 1990 period: str = AVAILABILITY_PERIOD, 1991 report: str = "ping4", 1992 device: str | None = None, 1993 group: str | None = None, 1994 ) -> list[dict[str, str]] | None: 1995 """ 1996 Retrieve the up and down event pairs behind a device's availability. 1997 1998 Where device mode gives the totals, this gives the outages that 1999 produced them, one row per pair. 2000 2001 # output format: {parent},{child},{down},{up},{total time},{match time} 2002 # example: nm-availability mode events time last1M report ping4 entity cisco-131-16-1 2003 2004 cisco-131-16-1,ping4,1603822871,1603822916,2389764,2388341 2005 cisco-131-16-1,ping4,1603088563,1603089823,2389764,2388341 2006 cisco-131-16-1,ping4,1603060380,1603060498,2389764,2388341 2007 2008 'down' and 'up' are epoch seconds bounding a single outage, so a 2009 device that went down twice comes back as two rows. Both are empty 2010 for a device that stayed up, which still reports the window it was 2011 measured over. 2012 2013 Take the length of an outage as 'up' minus 'down'. 'total time' and 2014 'match time' describe the measurement rather than the row they sit 2015 beside: every row in a reply carries the same 'total time', the 2016 length of the window, and a device's 'match time' is that less the 2017 time it spent down. Measured against a live server, a device with 2018 outages of 44 and 46 seconds came back with a 'match time' 90 below 2019 'total time' on both of its rows, while devices in the same reply 2020 that stayed up had the two equal. 2021 2022 So 'total time' is not the length of the outage on its row. Reading 2023 it that way gives the whole measurement window as the duration of a 2024 one minute flap. 2025 2026 These columns are not the ones group or device mode returns, so the 2027 three modes are parsed separately rather than sharing a field list. 2028 2029 Args: 2030 period (str): time filter, refer to the AKiPS programming guide 2031 (default: 'last24h'). 'lastNd' is calendar relative, meaning 2032 N-1 whole days plus today so far, so 'last1d' is today rather 2033 than 24 hours and shrinks to minutes just after midnight. 2034 'lastNh' and 'lastNm' are rolling windows of the length they 2035 name. 'total time' in the reply is the window measured 2036 report (str): 'ping4', 'ping6', 'snmp' or 'ifstatus', in any 2037 combination, comma separated (default: 'ping4') 2038 device (str): device to filter by, as '{device}' or 2039 '{device} {child}'. This is the device's AKiPS name 2040 exactly, its one primary key, which is either its sysName or 2041 its IP address depending on how the server names devices, and 2042 takes no pattern; get_device_by_ip() resolves an address to it 2043 group (str): group name to filter by 2044 Returns: 2045 A list of dictionaries, one per up and down pair, or None if 2046 nothing matched 2047 Raises: 2048 ValueError: if neither device nor group is given 2049 AkipsError: if the AKiPS server returns an error 2050 """ 2051 # Same as device mode, confirmed against a server: without a scope the 2052 # reply is empty rather than an error, which would arrive as None and 2053 # read as 'no outages' 2054 if device is None and group is None: 2055 raise ValueError( 2056 "get_event_availability needs a device or a group to scope it, " 2057 "an unscoped call returns nothing at all" 2058 ) 2059 params = { 2060 "maintenance": "off", # 'on' or 'off', show/hide maintenance mode devices 2061 "mode": "events", # 'group', 'device' or 'events' 2062 "time": period, # time filter, refer to programming guide 2063 "report": report, # 'ping4', 'ping6', 'snmp', 'ifstatus'. Any combination, comma separated, 2064 # 'entity' is not in the nm-availability syntax the AKiPS API 2065 # guide publishes, which lists only mode, time, report, group and 2066 # profile. It works, and is how device and event mode are scoped 2067 # here, but being undocumented it is the parameter most likely to 2068 # change under us in a future AKiPS release. 2069 "entity": device, # {device} [{child}] to filter by device or child 2070 "group": group, # {group name} to filter by group 2071 } 2072 text = self._get(section="api-availability", params=params) 2073 if text: 2074 # This endpoint sends no header row, so the column names come from 2075 # here rather than from the reply 2076 column_headers = [ 2077 "parent", 2078 "child", 2079 "down", 2080 "up", 2081 "total time", 2082 "match time", 2083 ] 2084 csv_to_list = self._parse_csv(text, fieldnames=column_headers) 2085 logger.debug("Found {} entries".format(len(csv_to_list))) 2086 return cast("list[dict[str, str]]", csv_to_list) 2087 return None
Retrieve the up and down event pairs behind a device's availability.
Where device mode gives the totals, this gives the outages that produced them, one row per pair.
output format: {parent},{child},{down},{up},{total time},{match time}
example: nm-availability mode events time last1M report ping4 entity cisco-131-16-1
cisco-131-16-1,ping4,1603822871,1603822916,2389764,2388341 cisco-131-16-1,ping4,1603088563,1603089823,2389764,2388341 cisco-131-16-1,ping4,1603060380,1603060498,2389764,2388341
'down' and 'up' are epoch seconds bounding a single outage, so a device that went down twice comes back as two rows. Both are empty for a device that stayed up, which still reports the window it was measured over.
Take the length of an outage as 'up' minus 'down'. 'total time' and 'match time' describe the measurement rather than the row they sit beside: every row in a reply carries the same 'total time', the length of the window, and a device's 'match time' is that less the time it spent down. Measured against a live server, a device with outages of 44 and 46 seconds came back with a 'match time' 90 below 'total time' on both of its rows, while devices in the same reply that stayed up had the two equal.
So 'total time' is not the length of the outage on its row. Reading it that way gives the whole measurement window as the duration of a one minute flap.
These columns are not the ones group or device mode returns, so the three modes are parsed separately rather than sharing a field list.
Arguments:
- period (str): time filter, refer to the AKiPS programming guide (default: 'last24h'). 'lastNd' is calendar relative, meaning N-1 whole days plus today so far, so 'last1d' is today rather than 24 hours and shrinks to minutes just after midnight. 'lastNh' and 'lastNm' are rolling windows of the length they name. 'total time' in the reply is the window measured
- report (str): 'ping4', 'ping6', 'snmp' or 'ifstatus', in any combination, comma separated (default: 'ping4')
- device (str): device to filter by, as '{device}' or '{device} {child}'. This is the device's AKiPS name exactly, its one primary key, which is either its sysName or its IP address depending on how the server names devices, and takes no pattern; get_device_by_ip() resolves an address to it
- group (str): group name to filter by
Returns:
A list of dictionaries, one per up and down pair, or None if nothing matched
Raises:
- ValueError: if neither device nor group is given
- AkipsError: if the AKiPS server returns an error
OUTPUT_FORMATS =
('raw', 'lines', 'key_value', 'attributes', 'csv', 'csv_dict')
The reply shapes call() can parse.
def
call( self, command: str | None = None, section: str = 'api-db', params: dict[str, typing.Any] | None = None, output: str = 'raw', user: str | None = None) -> Any:
2101 def call( 2102 self, 2103 command: str | None = None, 2104 section: str = "api-db", 2105 params: dict[str, Any] | None = None, 2106 output: str = "raw", 2107 user: str | None = None, 2108 ) -> Any: 2109 """ 2110 Send an arbitrary request to any AKiPS web API section and parse the 2111 reply in one of the shapes AKiPS replies in. 2112 2113 This is the general purpose call for anything the specific methods do 2114 not cover. It parses with the same routines they use, so an ad-hoc 2115 query returns the same shape its dedicated method would. 2116 2117 Sections do not share a parameter vocabulary. api-db takes a command 2118 string, while api-script, api-msg and api-availability each take their 2119 own named parameters, so pass 'command' for the first and 'params' for the 2120 others. Passing both adds the command to the given parameters. 2121 2122 Output formats, and where each one occurs: 2123 2124 raw the reply unchanged, as a string 2125 lines a list of non-blank lines 2126 key_value '{key} = {value}' lines, as from mgroup 2127 attributes '{parent} {child} {attribute} = {value}' lines, as from 2128 mget, nested by parent, child, then attribute 2129 csv CSV rows as lists, for replies with no header row 2130 csv_dict CSV rows as dictionaries keyed by the header row 2131 2132 Args: 2133 command (str): command string for the api-db section, shorthand 2134 for params={'cmds': command} 2135 section (str): API section to call (default: 'api-db') 2136 params (dict): parameters for sections that take no command string 2137 output (str): one of the formats listed above (default: 'raw') 2138 user (str): force the 'ro' or 'rw' account, for a section 2139 whose requirement is not in SECTION_USERS, or a command 2140 needing more rights than its section usually does 2141 Returns: 2142 The reply in the requested shape, or None if the server returned 2143 nothing 2144 Raises: 2145 ValueError: if output is not a supported format, or if neither 2146 command nor params was provided 2147 AkipsError: if the AKiPS server returns an error 2148 """ 2149 # Check before making the request, so a bad argument fails the same way 2150 # whether or not the server returned anything 2151 if output not in self.OUTPUT_FORMATS: 2152 raise ValueError( 2153 "Invalid output value provided to call, expected one of {}".format( 2154 ", ".join(self.OUTPUT_FORMATS) 2155 ) 2156 ) 2157 if command is None and params is None: 2158 raise ValueError("call requires either a command or a params dictionary") 2159 2160 request_params = dict(params or {}) 2161 if command is not None: 2162 request_params["cmds"] = command 2163 2164 text = self._get(section=section, params=request_params, user=user) 2165 if not text: 2166 return None 2167 2168 if output == "raw": 2169 return text 2170 if output == "lines": 2171 return self._parse_lines(text) 2172 if output == "key_value": 2173 return self._parse_key_value(text) 2174 if output == "attributes": 2175 return self._parse_attributes(text) 2176 if output == "csv_dict": 2177 return self._parse_csv(text, header=True) 2178 return self._parse_csv(text)
Send an arbitrary request to any AKiPS web API section and parse the reply in one of the shapes AKiPS replies in.
This is the general purpose call for anything the specific methods do not cover. It parses with the same routines they use, so an ad-hoc query returns the same shape its dedicated method would.
Sections do not share a parameter vocabulary. api-db takes a command string, while api-script, api-msg and api-availability each take their own named parameters, so pass 'command' for the first and 'params' for the others. Passing both adds the command to the given parameters.
Output formats, and where each one occurs:
raw the reply unchanged, as a string
lines a list of non-blank lines
key_value '{key} = {value}' lines, as from mgroup
attributes '{parent} {child} {attribute} = {value}' lines, as from
mget, nested by parent, child, then attribute
csv CSV rows as lists, for replies with no header row
csv_dict CSV rows as dictionaries keyed by the header row
Arguments:
- command (str): command string for the api-db section, shorthand for params={'cmds': command}
- section (str): API section to call (default: 'api-db')
- params (dict): parameters for sections that take no command string
- output (str): one of the formats listed above (default: 'raw')
- user (str): force the 'ro' or 'rw' account, for a section whose requirement is not in SECTION_USERS, or a command needing more rights than its section usually does
Returns:
The reply in the requested shape, or None if the server returned nothing
Raises:
- ValueError: if output is not a supported format, or if neither command nor params was provided
- AkipsError: if the AKiPS server returns an error