Resin - Application Server Health System - Java Monitoring

Resin Health System
Beyond Java Monitoring and Server Monitoring
Health Checks, Watchdog and Snapshot Report

Caucho Home | Contact Us | Caucho Blog | Wiki | Application Server / Web Server Copyright (c) 1998-2012 Caucho Technology, Inc. All rights reserved. caucho , resin and quercus are registered trademarks of Caucho Technology, Inc.
Java EE Certied
Gartner names Caucho in "Cool Vendors in Platform and Integration Middleware"
Caucho Home | Contact Us | Caucho Blog | Wiki | Application Server / Web Server Copyright (c) 1998-2012 Caucho Technology, Inc. All rights reserved. caucho , resin and quercus are registered trademarks of Caucho Technology, Inc.
Resin Health System (RHS) Overview
Resin Health System (RHS) Goes Beyond Just Monitoring Server and JVM can respond to conditions with actions Actions can remediate problems If server about to go down
due to bug, denial of service, or spike RHS triggers diagnostics then restarts Resin Application Server keeps running
Caucho Home | Contact Us | Caucho Blog | Wiki | Application Server / Web Server Copyright (c) 1998-2012 Caucho Technology, Inc. All rights reserved.
RHS : Reliability and System Transparency
RHS born from need
Idea for RHS came from doing Resin support Thread lock? Can you do a thread dump when you see the problem? Running out of memory? Can you do a heap dump? How is your machine congured? What version?
What OS?
RHS By Engineers for Engineers
Major features of Resin Health System (RHS)
Ability to respond to problems Detect JVM and OS issues Avoid zombie processes Restarts Resin if there are major problems Internal monitoring
External Monitoring
Resin WatchDog Process Uses process control, socket connection and periodic ping to determine up time status
Advanced Reporting PDF
Post-mortem analysis Thread Dump/Log Dump Meters and Graphs Heap Dump
Resin Internal WatchDog Thread Watchers internal meters for problems Periodic Thread
RHS Tracks Metrics
Metrics are things like Available Memory, Number of Requests Per Minute, Garbage Collection Time, CPU Load, etc. Metrics can be graphed Tracks Historical Data for Trends Can determine Anomalies Can determine Trends Can compare current data with baseline data
VisualizaFon
You can view data that Health System collects
Resin Web Admin Watchdog Report
Post mortem PDF Report PDF Report you can generate anytime
Snapshot Report
Trigger: CLI, REST, Through Web Admin
RHS and Web Admin
RHS: Health Checks
RHS is highly congurable Similar to the Resin's "URL Rewrite" rules Rules are congurable
checks, conditions, actions
Internal Watchdog periodic checks
Watchdog process
Lightweight process : Used to stop and start Resin instances Can restart an instance if Java Monitoring / Server Monitoring / Health issue Parent process of Resin Server Opens socket to Resin Server Sends are-you-alive ping?
Watchdog Process
Watchdog Non Stop Mode
Watchdog Non Stop Mode
Resin is resilient If a Denial of Service or unexpected Spike or Bug knocks down JVM, Resin restarts Beyond that Resin can detect critical problems and do critical diagnostics so DevOps and Developers can get to root of problem Resin long been product of choice for embedded devices, network appliances and large deployments Non Stop mode makes Resin perfect for cloud deployments
Resin Watch-Dog
Watchdog Process
Resin Watch-Dog
Watchdog Process
Resin Watch-Dog
Starting Resin Process Ownership TCP Link
Resin
Watchdog Process
Resin Watch-Dog
Non-Stop Up State Process Ownership TCP Link
Resin
Watchdog Process
Resin Watch-Dog
Non-Stop Up State Process Ownership TCP Link
Resin
Watchdog Process
Resin Watch-Dog
Non-Stop Up State
Watchdog Process
Internal Watchdog Thread Inside of Resin
Watchdog Process
Resin Health System Watchdog Thread
Resin Process
Internal Watchdog Thread Inside of Resin
Watchdog Process
Resin Process
Internal Watchdog Health Thread
Runs inside of Resin Server Runs periodically
Collects data Collects baseline data

Executes series of checks Recheck failed conditions Perform actions when conditions are critical or fatal
Resin Java CDI / CanDI and Resin Conf based
RHS conguration extends Resin conguration le resin.xml RHS uses CanDI (Resins Java CDI)
create and update Java objects, XML tags exactly matches either a Java class or a Java property Use HealthSystem JavaDoc Use JavaDoc of the various checks, actions,
CanDI means classes and cong is in JavaDocs
Java Doc / XML conf of RHS
Startup delay : wait for baselined date before recording Period: how often to check metrics Recheck period: if some level has been crossed how often should RHS recheck to see if better
Types of Health Checks
Health Checks produce Status
Resin Checks and Responds
Health System AcFons
AcFons based on condiFon
Actions can be grouped If in critical state for two minutes perform group of actions Dump JMX values, Dump Threads, Dump Heap, CPU Prole, Restart If actions longer than 10 m, restart
Collect data needed to diagnose
When something goes wrong
Bug
Denial of Service Attack Application Bug Unexpected Spike
RHS collects metrics you need to diagnose problem Without collection, you are ying blind
Denial of Service Spike
AcFons beQer than just watching
Watch dog report (PDF)
Post Mortem Report Environment Info Server Metrics JVM Metrics Thread Dump Heap Dump Metrics Graph
Watch dog report (PDF)
Post Mortem Report Environment Info Server Metrics JVM Metrics Thread Dump Heap Dump Metrics Graph
Environment Data
Collect critical information about environment When, What OS, What version of Resin How did Resin startup And much more
Health Status
Status of Health Checks in Report
Recent Errors and Warnings
Recent Errors and Warnings
Anomalies
Health Checking stores baseline Anomalies are congurable triggers based on large changes from expected baseline Anomaly detection is congurable can trigger actions
Understanding Anomaly DetecFon
Understanding Anomaly DetecFon
Types of Metric Graphs in Report
Cluster Status Request Count Request Time HTTP Request Errors Log Warnings Threads CPU Usage Database Connection Active
Database Query Time NetStat JVM Memory
Heap Used Tenured Used PermGen Used
GC Time
Sample Graphs Memory and GC Time
Sample Metric Graphs Request
GC and Memory Metrics
Heap Dump
Heap dump critical for tracking down memory leaks Also generates hprof le which can be analyzed by many third party tools
CPU Prole / Thread Dumps
Critical for debugging thread deadlock issues
Snapshot report
Reports same type of data as watchdog Watchdog report is a postmortem analysis Snapshots are whenever you feel like
e.g., during a stress test trigger via REST, CLI and Web Admin
Conclusion
More Background Info About Health System
Resin Health System : Java Monitoring and Server Monitoring built into Resin Application Server Resin Health System : Current and Into the Future Resin Application Server Fullls Vision of Cloud Computing Resin Health System Enhancements
More Info
Caucho Technology | Home Page Resin | Application Server Resin | Java EE Web Prole Application Server Resin - Cloud Support | 3G - Java Clustering Resin | Java CDI | Dependency Injection / IoC Resin - Health System | Java Monitoring and Server Monitoring Download Resin | Application Server Watch Resin | Application Server Featured Video
Resin Java ApplicaFon Server

Resin - Application Server Health System - Java Monitoring

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Resin - Application Server Health System - Java Monitoring

Uploaded by

Copyright:

Available Formats

Resin Health System

Beyond Java Monitoring and Server Monitoring

Health Checks, Watchdog and Snapshot Report

Gartner names Caucho in "Cool Vendors in Platform and Integration Middleware"

Resin Health System (RHS) Overview

RHS : Reliability and System Transparency

RHS born from need

RHS By Engineers for Engineers

Major features of Resin Health System (RHS)

Advanced Reporting PDF

RHS Tracks Metrics

You can view data that Health System collects

Resin Web Admin Watchdog Report

Trigger: CLI, REST, Through Web Admin

RHS and Web Admin

RHS: Health Checks

checks, conditions, actions

Internal Watchdog periodic checks

Watchdog Non Stop Mode

Watchdog Non Stop Mode

Internal Watchdog Thread Inside of Resin

Internal Watchdog Thread Inside of Resin

Internal Watchdog Health Thread

Runs inside of Resin Server Runs periodically

Collects data Collects baseline data

Resin Java CDI / CanDI and Resin Conf based

CanDI means classes and cong is in JavaDocs

Java Doc / XML conf of RHS

Types of Health Checks

Health Checks produce Status

Resin Checks and Responds

Health System AcFons

AcFons based on condiFon

Collect data needed to diagnose

When something goes wrong

Denial of Service Attack Application Bug Unexpected Spike

Denial of Service Spike

AcFons beQer than just watching

Watch dog report (PDF)

Watch dog report (PDF)

Status of Health Checks in Report

Recent Errors and Warnings

Recent Errors and Warnings

Understanding Anomaly DetecFon

Understanding Anomaly DetecFon

Types of Metric Graphs in Report

Database Query Time NetStat JVM Memory

Heap Used Tenured Used PermGen Used

Sample Graphs Memory and GC Time

Sample Metric Graphs Request

GC and Memory Metrics

CPU Prole / Thread Dumps

Critical for debugging thread deadlock issues

More Background Info About Health System

Resin Java ApplicaFon Server

You might also like