If you have a website, you have logs. If you have an ECommerce website, you have more logs; more important logs. If you have a deeper online presence including applications, reporting, financials, integrated systems, and customer accounts, you have even more important logs.
This is not supposed to be scary. It's quite empowering, really, as long as you actually use them and use them for your strategic business advantage.
What are log files anyway?
Log files contain the raw, unaltered technical events that happen in computer systems every moment, from every request, transaction, success, and failure. Every web page you look at generates anywhere from 1, 10, or 100 log file entries depending on the depth on interactivity that page contains and what you're doing. If it's mostly text-based, like reading the news, this will be lighter than, say surfing for movies, compiling financial reports, or gaming which will contain many more requests that go into those logs.
For some people the idea of looking at logs will, at first, seem intimidating. Like any business data once you learn what is meaningful for you, you will begin to see relevants parts of the data as it affects your business.
Traffic
Starting simply all businesses online will have some form of analytical logs about their website visitor traffic. Many hosting account providers will include these tools such as Webalyzer or AW Stats. There are ok, but a little basic. At the very least they will provide you with a basic understanding of visitors and traffic increases and decreases.
Google Analytics has also been a digital household name for many years, providing still deeper insight into traffic, user key performance indicators, as well as providing tools to deeper assess targets, goals, sales, and much more with an increasingly rich user experience. It provides a good foothold for newbies, making the most of trying not to alienate new users, while giving power users and consultants deeper tools to take control of your numbers' meaning.
Financial
Accountants and bookkeepers have been working with their own form of log files for hundreds of years. Journal entries have been the financial world's version of log entries -- automated transaction events now pervasive in on and offline electronic payments and are now the precursor to Journal entries. For every electronic transaction, a journal entry will be created; it makes sense that your financial team is aware of where your transactions originate just as they are aware of managing journal entries when the need arises.
You want to know every nook and cranny of when, how, and why, customers payments work as well as don't work. When you review your merchant account and payment system (if you have one) logs, you gain insight into what users are trying to do when they pay you. Enhanced security requirements means that some of your customers may not understand the detailed needs of banks for ecommerce payments. They may legitimately be trying to pay you, but may be ignoring the need for accurate card account address information, or CVD number requirements.
Banks may decline based on these kinds of failures but customers may not truly understand why – usually customer-facing errors simply say "DECLINED". By reviewing your logs you will start to understand where your customer base may need additional guidance in your checkout process or payment-failure messaging. It's great branding to be helpful, reduces frustration for your customers, and helps you get paid faster.
A successful payment, for example, could look like this
Invoice ID => 123456 User ID => 123 Amount => 11.95 <?xml version="1.0" encoding="utf-8"?> <createCustomerProfileTransactionResponse xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="AnetApi/xml/v1/schema/AnetApiSchema.xsd"> <messages><resultCode>Ok</resultCode> <message><code>I00001</code><text>Successful.</text></message></messages> <directResponse>1,1,1,This transaction has been approved.,123456,Y,99999999999,123456,,11.95,CC,auth_capture,149999992,Santa,Claus,,123 Main St,NiceTown,ON,M2Z 1V4,CA,4165551212,,santa@notarealdomain.com,,,,,,,,,,,,,,CXXXXXXXXXXXXXXXXX22222222,,,,,,,,,,,,,XXXX9999,American Express,,,,,,,,,,,,,,,,, </directResponse> </createCustomerProfileTransactionResponse>
What are the key takeaways here? Firstly, when you ignore the content code that (for now) looks meaningless, you see real messages in English:
<text>Successful.</text>
this means .... that the transaction was successful (if you didn't get that already)
11.95,CC,auth_capture
this means that this was payment on credit card (that's the "CC") of $11.95. "auth_capture" is merchant-speak for an actual payment (there are other kinds which we won't get into in this article)
There are also additional invoice, card type, and authorization codes as you can see in plain text inline with the rest of the content-code.
That was a positive result, now let's look at a negative:
Invoice ID => 902798 User ID => 987 Amount => 55.10 <?xml version="1.0" encoding="utf-8"?> <createCustomerProfileTransactionResponse xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="AnetApi/xml/v1/schema/AnetApiSchema.xsd"> <messages><resultCode>Error</resultCode> <message><code>E00027</code><text>This transaction has been declined.</text></message></messages> <directResponse>2,1,4,This transaction has been declined.,,S,999999999, 123456,,55.75,CC,auth_capture,387413529, Santa,Claus,,123 Main St,San Francisco,CA,90217,US, 4165551212,,santa@notarealdomain.com,,,,,,,,,,,,,,0XX2DXXXXXXXXXXXXXXX2222222,,,,,,,,,,,,,XXXX899X,Visa,,,,,,,,,,,,,,,,, </directResponse> </createCustomerProfileTransactionResponse>
Just like above the key takeaways here:
<text>This transaction has been declined.</text><resultCode>Error</resultCode>
If you hadn't guessed, this is a declined transaction and considers this an error. Important note: this doesn't say why just that it was declined. This could mean many things including fraud filtering, failure to use correct credit card number, or CVD number, and yes, it could have been declined for lack of funds.
<code>E00027</code>
This is the merchant code provided by your merchant company. It will not necessarily look like this, but merchants all have their transaction codes -- which you should have easy access to -- that will provide more details as to what this decline was caused by. This is a very valuable tool to have in your customer service toolbelt. Why? Because you can provide more insight for your management team and customer service team to relay and assist customers who are having trouble. Rather than just telling them that their payment was declined, you can provide more clarity and help which is tremendously powerful for situations where customers simply don't realize that correct addresses matter. Or that they got their CVD number wrong. You help, they understand, they correct, you get paid, and customer is happy far faster than having to phone their bank and wait.
123456,,55.75,CC,auth_capture,387413529
As before, this shows that the customer tried to pay (auth_capture) $55.75 on the credit card ("CC"), but failed as indicated by the error message.
Why learn these basics of eCommerce merchant logs? Because approvals and declines will always happen. Understanding the pattern of daily transaction results for your business is a powerful way to see if something may be affecting that pattern so you can take action as quickly as possible if you need to. Knowing that there are declines is one thing. Understanding why, how many are to be expected, and where they are coming from puts you heads and shoulders above the unknown. That's business management and control.
Have an understanding of what fluctuations look like. Failing to do so puts you in a position of being in the dark.
Security
Security will only increase in significance in the coming years. Your traffic, payment, as well as server error logs, will all show indicators of failures, but specifically failures where someone, or something (bots), may be trying to pull one over on your digital business.
Some generic hosting account types may not be equipping you with the security tools you need. Do you have firewall management? Can you isolate administrative back-end directories or addresses? Is root access open to the public? Do your payment systems treat all paying customers globally the same way regardless of their true IP address versus their credit card billing address?
Some merchant accounts may come with generic security "settings" that are, well, simple. Your merchant company, however, gets tense if they get an increase in chargebacks they perceive as due to fraud, or hack attempts – they will expect that you are on your game and aware of what the situation is, and security stapes to harden your processes.
For all of this, you gain immediate understandings when you know your logs.
Security logs will come in all different shapes and sizes based on what is being secured and what company platform created the system, but in general, you'll see events like this:
Brute Force attempt against account “santaclaus”.
This example uses the old fashioned Brute Force attack to illustrate. But this could include many different security event types including SQL Injection, Cross-Site Scripting, DDOS (Denial of Service)
A device at the “192.168.0.1” IP address has made a large number of invalidlogin attempts against the account “santaclaus”.This brute force attempt has exceeded the maximum number of failed loginattempts that the system allows. For security purposes, the system has temporarily blocked this IP address in order to prevent further attempts.
Service: nginx
This represents a website application attack (Nginx, or could be apache), but could include various service names you should have a basic familiarity with, including email service names such as dovecot. This is also something to consult with your Digital Business IT team on. These are easy to understand once you know what the naming is.
Local IP Address: 172.16.0.1
This is the last logged IP address of your account that the event used within your servers/accounts; if you have more than one this gives you a clue as to what internal place the user or user activity was trying to go
Local Port: 2096
The specific internal port used. Your applications will use different ports; hackers know this so will specifically look for ports that you may have left open that they can get into. Think of them as lots of little doors separate from your website's front door. You want to make sure they're locked and secured so that only the right software, systems, and users can access them.
Remote IP Address: 11.22.33.44
This is, theoretically, where the attack came from. I say "theoretically" because hack attempts are often hidden behind VPNs which obscure the real IP address. It is a good start, however, to know this.
Remote Port: 56652
Similar to the above, the specific port the user tried to get into. Your applications will use different ports; hackers know this so will specifically look for ports that you may have left open that they can get into. Think of them as lots of little doors separate from your website's front door. You want to make sure they're locked and secured so that only the right software, systems, and users can access them.
[Mon Jun 24 11:08:01.420182 2019] [authz_core:error] [pid 098765] [client 192.168.0.1:99947] AH01630: client denied by server configuration:/somevolumename/someaccountname/pathto/yourrootapplication/docs/changelog.txt
You will generally see security log events written out in formats like this. Date, time, error, code, English pseudo-description, and location of where this event took place.
As with the above example, this shows some basic information on date-time, type of error and where the event takes place in your systems such as changelog.txt in this case.
Security logs come in a great many forms and formats. You will find a wide array of types of logs for a great many types of systems such as email, website, eCommerce, account management, customer records, databases, payment systems, and almost anything else you can think of.
Big Picture
Once you're up and running, most of the time your logs will tell you what you likely already know. You will understand general patterns of successes and fails, which will in turn provide an understanding of where things could change for the better, as well as if things change for the worse.
As with any business KPIs (key performance indicators), you need to be able to speak to what your baseline performance is. Things change globally, digitally, constantly, and some of these events will affect you so you will want to have an understanding of what fluctuations look like. Failing to do so puts you in a position of being in the dark. If you rarely have any payment declines coming in, and then one day you get 10, what does this mean? If your page bounce rate is normally around 30% and quickly climbs to 50% is it urgent? If you find failed login attempts on your back end admin system increasing substantially, is it a cause for concern?
Most of these things are common enough in day to day for most businesses it must be told. That doesn't mean you have to love it, just acknowledge the meaning behind it.
The Magic
Once you get comfortable with your logs, give yourself a month to spend weekly time getting familiar with all your numbers and patterns, then you can start looking at strategies on how this information can help your business. Clearly this article is not about how to use every type of security log, analytics tools, BI platform, or financial rolling report, but a nudge to be aware that these are all-powerful and required elements of your business. Once you cultivate that comfort level you will gain a greater sense of understanding and control. A downtick in one log won't mean the end of the world and some extra failed login attempts from a certain network or country won't send you off the deep end. Why? Because you understand what your logs mean.
Whether your numbers improve or decline, knowing your logs gives you an understanding of the heartbeat, blood pressure, and cholesterol levels of your digital and physical business (they're all one now didn't you know?). With that your sense of control in good times and in not so good times, and that's far better than out of control.
