Posted in Industry News, WebSitePulse News
As you probably know, a number of news sources, corporations, and the OpenSSL team reported yesterday 14 October 2014 that version 3 of Secure Sockets Layer (SSLv3) is vulnerable at the protocol level. More information about the vulnerability can be found here - CVE-2014-3566.
To prevent any potential leaks from this vulnerability we have immediately disabled SSLv3 on all our web servers including the API endpoints. Our monitoring agents are not affected by this change and will continue to support SSLv3 for the time being in order to be able to monitor properly servers that do support SSLv3 only. We are urging all customers to disable SSLv3 on hosts interacting with the our services as soon as possible and upgrade to use Transport Layer Service (TLS).
Here are a few samples how to configure your potentially vulnerable services and disable SSLv3.
Change all SSLProtocol directives in your httpd config to
ALL -SSLv2 -SSLv3
and restart the server.
Add/edit the the following text to your server directive
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
If you can't disable SSL 3.0 entirely, there is TLS_FALLBACK_SCSV patch that can help avoid the attack, if both the client and the server support it.
Posted in Monitoring
Posted in Tech
Andrew Pruski is an SQL Server DBA currently working for Ding in Dublin, Ireland. He’s previously worked for Occam-DM Ltd and the United Kingdom Hydrographic Office in England. He’s benefitted immensely from the online DBA community, and his blog is an attempt to give something in return.
Aside from the SQL Server, he’s keen on running and playing rugby. He can also play the guitar (badly, but it doesn’t stop him).
How come you got involved in IT?
I always enjoyed working with computers when I was in school and so it seemed a natural area to study at the university. I enjoyed studying database design so once I completed my BSC in 2005, I started looking for a database developer position.
Whilst working as a junior developer, I realised that the code I was writing took a long time to execute (mainly because of all the cursors I was using). I did some research online and found multiple websites dedicated to SQL Server and opportunities to work as a DBA in particular. I read tons of online articles on websites such as SQLServerCentral.com and the more I read, the more interested I became in that area. It was then that I decided to pursue a career as a DBA.
What are the most common and the most challenging issues you've handled recently?
The most challenging issue I have ever had to perform was a task given to me when I was a relatively inexperienced DBA. A production data warehouse had a different collation to the development and staging versions and I was tasked with bringing the production database in-line with the other environments.
Why not change the collation of the databases in the other environments? Well, as I said, I was inexperienced and looking to prove myself, so off to work I went. The collation of a database can be changed quite simply from the Properties menu. The problem is that the setting only changes the collation for all new objects created. Everything else in the database remains on the old collation.
So after what felt like an age…I came up with the following procedure to change the collation of all the current objects with the database:
- Script out all database objects
- Drop all functions
- Drop all views
- Drop all foreign keys
- Drop all primary keys
- Drop all indexes
- Drop all statistics
- Change column collation
- Change database collation
- Recreate indexes
- Recreate primary keys
- Recreate foreign keys
- Recreate statistics
- Recreate views (and associated indexes)
I tested and tested, and tested, so when it came to deployment, it all ran smoothly. However, I’ll always think of this project as one of the most challenging I’ve worked on, not just because of the technical side but because of the human aspect of it as well. It taught me a lot about relationships within an SQL Server database but it also taught me that part of a DBA’s job should be to question why something is being done and to provide alternatives where possible. I should have questioned why the need to change the collation of the production database was felt. It would have been simpler (and a lot less risky) to change the development databases’ collation.
The most common issue that I deal with are blocking/deadlocks occurring within production databases. There is monitoring in place which sends out alerts (via email) meaning I can respond quickly to resolve any issues. An SQL Server agent polls active connections to check for blocking and Service Broker utilising Event Notifications has been implemented to generate deadlock alerts. This allows me to capture the offending query and see what action needs to be taken in order to prevent future incidents (more often than not, either rewriting the query or creating an appropriate index).
However, preventative measures are always better than having to react to issues in real time. I work closely with developers to ensure these issues are kept to a minimum by implementing good database design and by writing efficient SQL queries.
Do you use any external tools for database monitoring? Which ones?
I have worked with Nagios, Zabbix, RedGate, Idera and Confio monitoring tools. All are good systems with their own positives and negatives but I strongly feel that any DBA should be able to implement their own monitoring system. I wrote a short blog post detailing my reasons why.
In your opinion, what server status variables should a DBA log and monitor?
I always create my own database in any SQL instances that I administer. This allows me to setup monitoring for the following:
- Failed SQL Server Agent jobs
- Percentage transaction log used
- Corruption alerts
- Wait statistics
- Index usage statistics
- Index fragmentation
- Error log auditing
- Disk space
- Memory Usage
- Page life expectancy vs. lazy writes
- IO subsystem performance.
What about disaster recovery and prevention? What is your approach?
Every disaster recovery plan should be tailored to each individual system. It is one of the key aspects of a DBA role within the company to work closely with system owners to define Recovery Point Objectives (how much data can be lost it the event of a failure) and Recovery Time Objectives (how long it should take to bring the system back online).
Once the RPO & RTO have been established, a high availability solution can be built and a disaster recovery strategy implemented. High availability covers SQL Server features such as mirroring, clustering and always on. Disaster recovery strategies start with a backup schedule for the databases but can include features such as log shipping or asynchronous mirroring to a DR site.
I attended an SQL Skills training course last year in which Paul Randal talked about how to approach backup strategies. His advice was to think of backup strategies as restore strategies and that DBAs should consider performing many restores to bring the database online in the event of a system failure. This is one of the best pieces of advice I have been given as a DBA. It is all well and good to back up database transaction logs every 15 minutes, but how many of those will need to be restored in the event of a failure? I really do not want to be restoring hundreds of log backups at 3 o’clock in the morning (because this is generally when failures happen for me, for some reason).
Contact him at:
Posted in Monitoring, Industry News, Tech
facebook.com went down for aboutt 15 minutes between 4:00 and 4:15 AM EST. Users trying to connect to the site were seeing an error message "Sorry, something went wrong". The issue has been confirmed by multiple locations around the world and as far as we can tell all Facebook users were affected.
The Facebook Platform Status showed a sharp increase in API response times for the time of the outage.
The last information posted by Facebook is
Earlier this morning, we experienced an issue that prevented use of the API for a brief period of time. We resolved the issue quickly, and we are now back to 100%. We're sorry for any inconvenience this may have caused.
We are awaiting an official statement and more details from Facebook and we will update this post as soon as more information is available.
Posted in Monitoring
According to a survey conducted by the National Retail Federation, the total spent for Mother’s Day this year reached $20 billion.
Considering the popularity of this day, and the growing popularity of online shopping, we couldn’t stress more on how important it is for online stores to maintain flawless performance and availability during the weeks prior to Mother’s Day. Just like a bad selection of gifts and high prices could turn visitors away from your website, a slow website, inability to perform an online purchase or website downtime could be just as harmful, sending your potential buyers to competitors’ websites.
As usual, we monitored 12 of the most popular websites for purchasing gifts and flowers for moms. We were specifically observing their uptime and response time as the most important metrics for the reliability of a website.
Response time varied between 1,4 (personalizationmall.com) and 18,4 seconds (giftbasketsplus.com). This means that some of the websites were not able to bear the heavy traffic during a certain period of time. That, on the other hand, could have resulted in disappointed buyers and revenue loss.
At the same time, uptime varied between 100% (gifttree.com, sees.com, proflowers.com, fromyouflowers.com, beyondblossoms.com) and 99.7% (giftbasketsplus.com) which is quite acceptable.
For more details, look at the infographic above or go to websitepulse.com.