Quicken's algorithm for detecting "Matches" vs. "New" transactions needs work

Hi all:

Struggling through the process of activating One Step Update for my various Schwab accounts frequently leaves me with the feeling that the Quicken software coders aren't among the sharpest tools in the shed and that Quicken's QC leaves a lot to be desired. But, maybe I'm missing something.

So, I activated my Schwab Investor Checking account for OSU. More than 380 transactions got downloaded, of which Quicken decided that exactly 9 transactions were "Match" transactions with the balance of the transactions being "New."

Here's a transaction that Quicken declared to be a Match in the Downloaded Transactions window:

8/3/2009 "Blue Shield of California"/BLUE SHIELD CA BlueShield $XXX.XX

and here's how it appears in my register:

8/1/2009 Blue Shield of California $XXX.XX

The amounts are the same in both cases.

Here's a transaction that Quicken declared to be New in the Downloaded Transaction window:

9/1/2009 "Blue Shield of California"/BLUE SHIELD CA BlueShield $XXX.XX

and here's how it appears in my register:

9/1/2009 Blue Shield of California $XXX.XX

Again, the amounts are the same in both cases.

It's clear that the "New" transaction is a better match to the register entry than the "Match" transaction - the dates are different in the Match while the dates are the same in the New - so I'm curious as to what's going on behind the curtain (i.e., in the programming) to come up with these anomalous results.

Tom Young

Reply to
TomYoung
Loading thread data ...

Quicken matches my manually entered transactions almost flawlessly. I can't remember the last time a transaction was mismatched.

Reply to
Bob Wang

I can, it was today. Only one today, several hundreds in the past.

Reply to
XS11E

TomYoung wrote in news: snipped-for-privacy@u4g2000prn.googlegroups.com:

You're not, and I don't even need to read the rest of your message to see what you're complaining about.

The product has gone completely downhill since they moved development overseas.

Like that's shock.

My grandmother could code better and test better, and she's dead.

Reply to
Eric J. Holtman

[Talking about non-investment accounts.]

No it doesn't work that way. And the fact that you think the cleared status should play a part in the process shows that you don't understand what the "match" process is all about.

First: The match process is *only* about matching downloaded transactions with transactions that have not been downloaded**. Cleared status is not a valid indicator of whether a transaction has been downloaded. Cleared status can be set without regard to whether a transaction has ever been downloaded.

Second: the downloaded date is NOT a "transaction" date; it is a "cleared" date. But the date of a manually entered transaction in your Quicken register - especially for check transactions - is the transaction date ... the date you wrote the check. Checks written two weeks ago can easily clear before checks written 2 days ago. Comparing "cleared date" to "transaction date" is an unreliable comparison. Your date comparison theory fails miserably.

Third: in many cases, the user can prevent Quicken from matching to an incorrect transaction, by "telling" Quicken that the existing transaction has been downloaded before ... even though it may not have been. You can do this by right-clicking the register transaction, then holding down CTRL while left-clicking "Copy transaction(s)". Then putting a check mark in "Downloaded Transaction" and selecting a "Posting Date". Doing that should exclude that transaction from ever being a candidate for matching*.

Fourth: Actually, I believe the match process does give weight to the "date". But, you don't seem to understand that downloaded transactions can represent transactions whose Quicken "transaction date" was well in the past (see #2). And there can have been, and often are, many other transactions for the same payee, which also have not been downloaded before ... the fact that they have newer "dates" doesn't insure that they are the "right" transaction to match to (generally speaking; it's better to have older transactions legitimately cleared, than newer transactions).

I do believe that Intuit might be able to tweak the matching process and improve it *very* slightly (at some questionable cost) ... but it will never be able to be what you seem to expect it to be.

Can you provide evidence that some other product has done a measurably better job? Remembering that the product must produce results that provide equal benefits to other customers besides yourself?

[** If a transaction marked as already downloaded, is treated as a match candidate ... I suspect it is a file corruption problem. It has never happened to me; and I've never read any proof that this happened to anyone else ... where corruption could not be the problem.]
Reply to
John Pollard

I don't necessarily think the cleared status should play a part in the process, I was simply speculating that it did since that was the only other difference between the two mentioned transactions. Oh, that and the glaring discrepancy in the dates, which Quicken seemed to ignore in its calculation of the most likely match.

You're chasing a red herring. I didn't say cleared status was a vital aspect of the match process, I only speculated that Quicken thought it was, and maybe I was wrong.

(Man, I don't just "fail", I fail "miserably!" :-) )

Logically, for checks, cleared date comes after the date entered in the register. The August 3, 2009 date downloaded by Schwab is the date the payment was deducted from the checking account. It's the true "cleared" date and has nothing to do with the date - probably not recorded by Quicken - that a "c" or "R" gets entered into the Clr column. By that logic alone, a logic easily programmed, the Match should never have lined up on the 8/1/10 register date (issue date) since it was almost a year after the 8/3/09 cleared date. Any human being looking at a check to Blue Shield in the amount of $XXX.XX cleared 8/3/09 would have guessed that transaction matched to the

8/1/09 payment of $XXX.XX to Blue Shield. It might not have, but that would be the best guess.

I guess that's good to know and I'll try to remember that if I run into it again. I was simply astonished that Quicken couldn't do a much better job at guessing at proper matches.

Since almost all my transactions in the checking account are EFT's, 99 times out a 100 the register date and the date downloaded were exactly the same, for the same amount, with Payee names that were very, very similar. Quicken came up with an immaterial amount of matches (9 out of well over 300), and as you can see from the one example I posted, some of the matches didn't make any sense.

I completely understand the difference between dates and how they add some uncertainty to the process. I really don't expect perfection, I just think there's lots of room for improvement in coming up with "Matches" that turn out to be correct. Since virtually all my payments are EFT's maybe, in a way, I make it more difficult for Quicken since it can't look for check numbers (assuming it even uses check numbers) and there's at least a metaphysical possibility that the EFT I record on 8/1/08 doesn't clear until 9/1/08.

However, going back to my monthly Blue Shield payment; every month I schedule that EFT to go out on the 1st of the month; the payment has been exactly the same for all that time; if the 1st isn't a business day the payment gets made on the next business day following. Any human being looking at the downloaded information would have very confidently lined up the Blue Shield payments with the correct register dates, resulting in 24 or 25 proposed "matches." As it was, Quicken gave me one "match" that didn't make a lick of sense and 23 or

24 "new" transactions that it wanted me to match manually.

Up until this post I haven't really said what I expected it to do, I was simply astonished how poorly is does what it does. I think it can do better. I don't think programming to come up with high-probability matches in a personal finance check register is mind-boggling hard. I don't expect perfection.

I've stated that, in my opinion, Quicken's matching algorithm does a poor job. I've stated that coming up with high-probability matches for a personal checking account (25 - 30 payments a month, half a dozen deposits?) doesn't appear to be a mind-boggling logical task. Accordingly, programming this logic isn't as daunting an undertaking (or as costly) as another Manhattan Project. I've stated that perfection isn't required. I believe that any Quicken of the millions of Quicken customers (current and future) attempting to activate OSU for such an account and faced with 300+ transactions on the initial download would benefit from this improvement. I may be wrong, but that all seems reasonable to me.

Since I haven't used other personal finance software I can't "provide evidence that some other product has done a measurably better job", but I hope you're not suggesting that until I do I can't reasonably criticize Quicken, are you?

Tom Young

Reply to
TomYoung

Quicken does take the check number field into account, though I've forgotten the exact rules (they used to be available at Intuit's kb web site, but I haven't seen them for a long time). I think that the downloaded transaction and the register transaction should have the same value (including spaces) in the Num field.

I have several EFT's every month; Quicken always matches them correctly.

A human being can look at another human being whose face they've seen before and easily recognize the face ... writing software to do facial recognition is not a trivial exercise.

Basically, a one-time benefit ... and only for some users.

The first download (or the first several downloads) to a Quicken account that has been active for some time, is the most difficult time for getting the New/Match status correct. Those who began downloading at the same time they first began using Quicken, shouldn't face the same problem.

I'm saying that if you make a claim - such as that it is economically feasible to provide a significantly better matching algorithm - you need to provide verifiable evidence of that claim. Pointing to some other product that does a better job would be one way to demonstrate your point.

But, like Bob Wang, I do not have any problems, like yours, with my downloaded transactions. I'd say Quicken gets my New/Match status right about 99% of the time (I can't remember the last time Quicken got it wrong, but I'm allowing for a 1% margin of error on my part); I'm hard pressed to imagine seeing it get much better than that. [I download quite a few accounts from several financial institutions.]

Reply to
John Pollard

TomYoung wrote in news: snipped-for-privacy@q21g2000prm.googlegroups.com:

Well, what did you expect, when you tried to argue with a fundamentalist?

Reply to
Eric J. Holtman

I didn't say it was a direct or perfect analogy. But you were the one who insinuated that since a human could visually do it, a program could do it as easily. My point was that there are many things that humans can do much more easily than computers; it's not meaningful to note how easily humans can do it ... it only matters (to this discussion) how easily a computer can be programmed to do it. That's not so hard to understand, is it?

You're straining so hard, you mis-state that which you quoted. I said "Pointing to some other product that does a better job would be *one way* to demonstrate your point". [Emphasis added.] "One way" is clearly not the same as the "only way".

Apparently you want to make unsubstantiated claims ... which no one should disagree with ... and you're not to be expected to provide anything other than your personal opinion, which is to be taken as fact.

It's trivial to state that some task you want performed is trivial (and cost effective). Anyone can say that about anything they want, anytime. Why not believe that all claims, not disproven, are true, even if they have no verifiable supporting evidence? Or is it just your claims that require no supporting evidence?

Suppose that it was intellectully trivial and virtually cost free to provide the matching algorithm that you believe Quicken should have; in other words, suppose it is as good a business idea as you would have everyone believe. What would be your opinion as to why it wasn't done long ago? Quicken programmers are so incompetent that they can't even perform such a trivial exercise? Intuit doesn't know how to run their own business? [But you do?] Intuit doesn't care about, or listen to, their customers? Intuit is evil?

Reply to
John Pollard

"John Pollard" wrote in news:i3mmi8$uk6$ snipped-for-privacy@news.eternal-september.org:

Yup. Have they fixed the "flashing bug" yet?

Yup.

Nope. Never attribute to malice that which is easily explained as stupidity.

Reply to
Eric J. Holtman

I'll apply that logic to your remarks.

Reply to
John Pollard

I'm a big Quicken fan. I use it daily, but I don't have a problem seeing that it has ugly warts.

Free software SBWizard does an outstanding job of handling US Savings Bonds. After decades, Intuit still can't handle this very widespread way of investing. Why?

Obviously, the programming for this task wasn't insurmountable. But, Intuit mucks with a pretty decent user interface every year, while leaving some very large holes in their ability to handle common investments.

I'd bet that many of the technical issues and limitations with Quicken could be resolved, or at least improved, but the management doesn't seem to have that as one of their priorities. They haven't for many years, at least as far back as the TurboTax DRM debacle.

They bought up or squeezed out their competition, using their established position in the industry, until they have a virtual monopoly on consumer financial software. Then, they sat on their laurels for a decade, issuing new releases with little more than window dressing.

-- Jim

Reply to
Jim H

Jim H wrote in news:o-

2dnZP4qI snipped-for-privacy@giganews.com:

Yup. I think I've used it daily since 1990 or 1991.

And I completely understand that. You've got people locked in, and the best way to maximize return for Intuit shareholders is to minimize expense.

Doesn't mean it doens't suck though.

Reply to
Eric J. Holtman

"John Pollard" wrote in news:i3n299$hk2$1 @news.eternal-september.org:

Dance, John, dance!!!

Reply to
Eric J. Holtman

BeanSmart website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.