Despite being in a potentially favorable situation, Hooker halted his brief offensive. His actions may have demonstrated his lack of confidence in handling the complex actions of such a large organization for the first time (he had been an effective and aggressive division and corps commander in previous battles), but he had also decided before beginning the campaign that he would fight the battle defensively, forcing Lee, with his small army, to attack his own, larger one. At the [First] Battle of Fredericksburg (December 13, 1862), the Union army had done the attacking and met with a bloody defeat.

Hooker knew Lee could not sustain such a defeat and keep an effective army in the field, so he ordered his men to withdraw back into the Wilderness and take a defensive position around Chancellorsville, daring Lee to attack him or retreat with superior forces at his back. He confused matters by issuing a second order to his subordinates to hold their positions until 5 p.m., but by the time it was received, most of the Union units had begun their rearward movements. That evening, Hooker sent a message to his corps commanders, "The major general commanding trusts that a suspension in the attack to-day will embolden the enemy to attack him."

Hooker's subordinates were surprised and outraged by the change in plans. They saw that the position they were fighting for near the Zoan Church was relatively high ground and offered an opportunity for the infantry and artillery to deploy outside the constraints of the Wilderness. Meade exclaimed, "My God, if we can't hold the top of the hill, we certainly can't hold the bottom of it!" Viewing through the lens of hindsight, some of the participants and many modern historians judged that Hooker effectively lost the campaign on May 1. Stephen W. Sears observed, however, that Hooker's concern was based on more than personal timidity.

The ground being disputed was little more than a clearing in the Wilderness, to which access was available by only two narrow roads. The Confederate response had swiftly concentrated the aggressive Stonewall Jackson's corps against his advancing columns such that the Federal army was outnumbered in that area, about 48,000 to 30,000, and would have difficulty maneuvering into effective lines of battle. Meade's two divisions on the River Road were too far separated to support Slocum and Sykes, and reinforcements from the rest of the II Corps and the III Corps would be too slow in arriving.

Although Hooker's left flank was firmly anchored by Meade's V Corps on the Rappahannock, and his center was strongly fortified, his right flank was "in the air." Howard's XI Corps was camped on the Orange Turnpike, extending past Wilderness Church, and was vulnerable to a flanking attack. Investigations of a route to be used to reach the flank identified the proprietor of Catharine Furnace, Charles C. Wellford, who showed Jackson's cartographer, Jedediah Hotchkiss, a recently constructed road through the forest that would shield marchers from the observation of Union pickets. Lee directed Jackson to make the flanking march, a maneuver similar to the one that had been so successful prior to the Second Battle of Bull Run (Second Manassas). An account by Hotchkiss recalls that Lee asked Jackson how many men he would take on the flanking march and Jackson replied, "my whole command."

Given the communications chaos of May 1, Hooker was under the mistaken impression that Sedgwick had withdrawn back across the Rappahannock and, based on this, that the VI Corps should remain on the north bank of the river across from the town, where it could protect the army's supplies and supply line. In fact, both Reynolds and Sedgwick were still west of the Rappahannock, south of the town.

Confederate cavalry under Stuart kept most Union forces from spotting Jackson on his long flank march, which started between 7 and 8 a.m. and lasted until midafternoon. Several Confederate soldiers saw the Union observation balloon Eagle soaring overhead and assumed that they could likewise be seen, but no such report was sent to headquarters. When men of the III Corps spotted a Confederate column moving through the woods, their division commander, Brig. Gen. David B. Birney, ordered his artillery to open fire, but this proved little more than harassment. The corps commander, Sickles, rode to Hazel Grove to see for himself and he reported after the battle that his men observed the Confederates passing for over three hours.

When Hooker received the report about the Confederate movement, he thought that Lee might be starting a retreat, but he also realized that a flanking march might be in progress. He took two actions. First, he sent a message at 9:30 a.m. to the commander of the XI Corps, Maj. Gen. Oliver O. Howard on his right flank: "We have good reason to suppose the enemy is moving to our right. Please advance your pickets for purposes of observation as far as may be safe in order to obtain timely information of their approach."

At 10:50 a.m., Howard replied that he was "taking measures to resist an attack from the west." Hooker's second action was to send orders to Sedgwick – "attack the enemy in his front" at Fredericksburg if "an opportunity presents itself with a reasonable expectation of success" – and Sickles – "advance cautiously toward the road followed by the enemy, and harass the movement as much as possible". Sedgwick did not take action from the discretionary orders. Sickles, however, was enthusiastic when he received the order at noon. He sent Birney's division, flanked by two battalions of Col. Hiram Berdan's U.S. sharpshooters, south from Hazel Grove with orders to pierce the column and gain possession of the road.

Although by now it was 3 p.m., Jackson decided to march his men two miles farther and turn right on the Turnpike instead, allowing him to strike the unprotected flank directly. The attack formation consisted of two lines—the divisions of Brig. Gens. Robert E. Rodes and Raleigh E. Colston—stretching almost a mile on either side of the turnpike, separated by 200 yards, followed by a partial line with the arriving division of A.P. Hill.

Also, the XI Corps was not well respected – an outfit with poor morale and no history of battlefield success. Many of its officers and enlisted men were immigrants from Germany and other parts of Central Europe, including a number of political refugees from the 1848 revolutions. The corps had been formed in the spring of 1862 by merging Brig. Gen Louis Blenker's division with Maj. Gen John C. Frémont's Mountain Department in West Virginia. After a miserable trek across Virginia in which Blenker's troops were provisioned inadequately and suffered from widespread hunger, disease, and desertion, they joined with Fremont in a campaign that resulted in them being soundly defeated by Stonewall Jackson. Jackson turned to Robert Rodes and asked him "General, are you ready?" When Rodes nodded, Jackson replied, "You may go forward then." Most of the men of the XI Corps were encamped and sitting down for supper and had their rifles unloaded and stacked. Their first clue to the impending onslaught was the observation of numerous animals, such as rabbits and foxes, fleeing in their direction out of the western woods. This was followed by the crackle of musket fire, and then the unmistakable scream of the "Rebel Yell".

Two of von Gilsa's regiments, the 153rd Pennsylvania and 54th New York, had been placed up as a heavy skirmish line and the massive Confederate assault rolled completely over them. A few men managed to get off a shot or two before fleeing. The pair of artillery pieces at the very end of the XI Corps line were captured by the Confederates and promptly turned on their former owners. Devens's division collapsed in a matter of minutes, slammed on three sides by almost 30,000 Confederates. Col. Robert Reily and his 75th Ohio managed to resist for about ten minutes before the regiment disintegrated with 150 casualties, including Reily himself, and joined the rest of the fleeing mob.

Col. Lee would later write sarcastically, "A rifle pit is useless when the enemy is on the same side and in rear of your line." Some men tried to stand and resist, but they were knocked over by their fleeing comrades and a hail of Confederate bullets. Maj. Gen. Carl Schurz ordered his division to shift from an east–west alignment to north–south, which they did with amazing precision and speed. They resisted for about 20 minutes and "Leatherbreeches" Dilger managed to drive the Confederates off the turnpike for a bit with his guns, but the sheer weight of Jackson's assault overwhelmed them, too, and they soon had to flee.

Dilger for a time stood alone with a gun firing double-shotted canister at the attackers, then limbered up to flee as the Confederates closed in on him. Three of his artillery horses were shot dead, and when he realized that the gun could not be moved, he had to abandon it. General Howard partially redeemed his inadequate performance prior to the battle by his personal bravery in attempting to rally the troops. He stood shouting and waving a flag held under the stump of his amputated arm lost at the Battle of Seven Pines in 1862, ignoring the danger of the heavy rifle fire, but he could only gather small pockets of soldiers to resist before his corps disintegrated. Col. Adolf Buschbeck's brigade put up a last-ditch stand along with Dilger's guns. They too had to retreat, but maintained good order as they went.

The chaos unfurling on the Union right had gone unnoticed at Hooker's headquarters until at last the sound of gunfire could be heard in the distance, followed by a panic-stricken mob of men and horses pouring into the Chancellorsville clearing. A staff officer yelled "My God, here they come!" as the mob ran to and past the Chancellor mansion. Hooker jumped onto his horse and frantically tried to take action. He ordered Maj. Gen Hiram Berry's division of the III Corps, once his own division, forward, yelling "Receive them on your bayonets!" Artillerymen around the clearing began moving guns into position around Fairview Cemetery.

Meanwhile, down at Hazel Grove, the 8th Pennsylvania Cavalry were relaxing and awaiting orders to chase after Confederate wagon trains, also oblivious to the collapse of the XI Corps. The regiment's commander, Maj. Pennock Huey, received a notice that General Howard was requesting some cavalry. Huey saddled up his men and headed west along the turnpike, where they ran straight into Robert Rodes's division. After a confused fight, the 8th Pennsylvania Cavalry retreated to the safety of the Chancellorsville clearing with the loss of 30 men and three officers.

Jackson's force was now separated from Lee's men only by Sickles's corps, which had been separated from the main body of the army after its foray attacking Jackson's column earlier in the afternoon. Like everyone else in the Union army, the III Corps had been unaware of Jackson's attack. When he first heard the news, Sickles was skeptical, but finally believed it and decided to pull back to Hazel Grove.

May 3: Chancellorsville

thumb|upright=1.5|Chancellorsville, actions on May 3, dawn to 10 a.m.

Despite the fame of Stonewall Jackson's victory on May 2, it did not result in a significant military advantage for the Army of Northern Virginia. Howard's XI Corps had been defeated, but the Army of the Potomac remained a potent force and Reynolds's I Corps had arrived overnight, which replaced Howard's losses. About 76,000 Union men faced 43,000 Confederate at the Chancellorsville front. The two halves of Lee's army at Chancellorsville were separated by Sickles's III Corps, which occupied a strong position on high ground at Hazel Grove.

Unless Lee could devise a plan to eject Sickles from Hazel Grove and combine the two halves of his army, he would have little chance of success in assaulting the formidable Union earthworks around Chancellorsville. Fortunately for Lee, Joseph Hooker inadvertently cooperated. Early on May 3, Hooker ordered Sickles to move from Hazel Grove to a new position on the Plank Road. As they were withdrawing, the trailing elements of Sickles's corps were attacked by the Confederate brigade of Brig. Gen. James J. Archer, which captured about 100 prisoners and four cannons. Hazel Grove was soon turned into a powerful artillery platform with 30 guns under Col. Porter Alexander.

Although Stuart was a cavalryman who had never commanded infantry before, he was to deliver a creditable performance at Chancellorsville. By the morning of May 3, the Union line resembled a horseshoe. The center was held by the III, XII, and II Corps. On the left were the remnants of the XI Corps, and the right was held by the V and I Corps. On the western side of the Chancellorsville salient, Stuart organized his three divisions to straddle the Plank Road: Heth's in the advance, Colston's 300–500 yards behind, and Rodes's, whose men had done the hardest fighting on May 2, near the Wilderness Church.

Rodes sent his men in last and this final push, along with the excellent performance of the Confederate artillery, carried the morning battle. Chancellorsville was the only occasion in the war in Virginia in which Confederate gunners held a decided advantage over their Federal counterparts. Confederate guns on Hazel Grove were joined by 20 more on the Plank Road to duel effectively with the Union guns on neighboring Fairview Hill, causing the Federals to withdraw as ammunition ran low and Confederate infantrymen picked off the gun crews.

Fairview was evacuated at 9:30 a.m., briefly recaptured in a counterattack, but by 10 a.m. Hooker ordered it abandoned for good. The loss of this artillery platform doomed the Union position at the Chancellorsville crossroads as well, and the Army of the Potomac began a fighting retreat to positions circling United States Ford. The soldiers of the two halves of Lee's army reunited shortly after 10 a.m. before the Chancellor mansion, wildly triumphant as Lee arrived on Traveller to survey the scene of his victory.

At the height of the fighting on May 3, Hooker suffered an injury when at 9:15 a.m. a Confederate cannonball hit a wooden pillar he was leaning against at his headquarters. He later wrote that half of the pillar "violently [struck me] ... in an erect position from my head to my feet." He likely received a concussion, which was sufficiently severe to render him unconscious for over an hour. Although clearly incapacitated after he arose, Hooker refused to turn over command temporarily to his second-in-command, Maj. Gen. Darius N. Couch, and, with Hooker's chief of staff, Maj. Gen. Daniel Butterfield, and Sedgwick out of communication (again due to the failure of the telegraph lines), there was no one at headquarters with sufficient rank or stature to convince Hooker otherwise. This failure may have affected Union performance over the next day and may have directly contributed to Hooker's seeming lack of nerve and timid performance throughout the rest of the battle.

Meanwhile, at about 7:30 a.m., near Chancellorsville, III Corps, Second Division commander Maj. Gen. Hiram Berry was killed by Confederate musket fire. Brig. Gen. Gershom Mott, next in seniority, was also severely wounded at about that time. Believing himself to be next in command, Brig. Gen. Joseph Warren Revere (grandson of Paul Revere) assumed command of the division. Finding himself in among stragglers from the battle and unable to contact Sickles, Revere commanded a group of 500 or 600 soldiers to reform at a point about three miles to the north of Chancellorsville. in which Jackson used horoscopes and astrology to predict Revere's "culmination of the malign aspect" in the first days of May 1863.

Lee had left a relatively small force at Fredericksburg, ordering Maj. Gen. Jubal Early to "watch the enemy and try to hold him." If he was attacked in "overwhelming numbers," Early was to retreat to Richmond, but if Sedgwick withdrew from his front, he was to join with Lee at Chancellorsville. On the morning of May 2, Early received a garbled message from Lee's staff that caused him to start marching most of his men toward Chancellorsville, but he quickly returned after a warning from Brig. Gen. William Barksdale of a Union advance against Fredericksburg.

John Sedgwick's road to Chancellorsville was open, but he wasted time in gathering his troops and forming a marching column. His men, led by Brooks's division, followed by Newton and Howe, were delayed for several hours by successive actions against the Alabama brigade of Brig. Gen. Cadmus M. Wilcox. His final delaying line was a ridge at Salem church, where he was joined by three brigades from McLaws's division and one from Anderson's, bringing the total Confederate strength to about 10,000 men.

The fighting on May 3, 1863, was some of the most furious anywhere in the civil war. The loss of 21,357 men that day in the three battles, divided equally between the two armies, ranks the fighting only behind the Battle of Antietam as the bloodiest day of war in American history.

May 4–6: Union withdrawals

thumb|upright=1.5|Chancellorsville, actions on May 4, withdrawals on May 5 and 6

On the evening of May 3 and all day May 4, Hooker remained in his defenses north of Chancellorsville. Lee observed that Hooker was threatening no offensive action, so felt comfortable ordering Anderson's division to join the battle against Sedgwick. He sent orders to Early and McLaws to cooperate in a joint attack, but the orders reached his subordinates after dark, so the attack was planned for May 4.

By this time Sedgwick had placed his divisions into a strong defensive position with its flanks anchored on the Rappahannock, three sides of a rectangle extending south of the Plank Road. Early's plan was to drive the Union troops off Marye's Heights and the other high ground west of Fredericksburg. Lee ordered McLaws to engage from the west "to prevent [the enemy] concentrating on General Early."

The attack finally began around 6 p.m. Two of Early's brigades (under Brig. Gens. Harry T. Hays and Robert F. Hoke) pushed back Sedgwick's left-center across the Plank Road, but Anderson's effort was a slight one and McLaws once again contributed nothing. Throughout the day on May 4, Hooker provided no assistance or useful guidance to Sedgwick, and Sedgwick thought about little else than protecting his line of retreat.

Confederate dead behind the stone wall of Marye's Heights, Fredericksburg, Virginia, killed during the Chancellorsville campaign (the [[Second Battle of Fredericksburg), May 1863. Photograph by A.J. Russell.|thumb|left]]

It was a difficult operation. Hooker and the artillery crossed first, followed by the infantry beginning at 6 a.m. on May 6. Meade's V Corps served as the rear guard. Rains caused the river to rise and threatened to break the pontoon bridges.

Aftermath

Casualties

{| class=wikitable

!bgcolor="#99ccff"|Senior officer casualties

|-

|<gallery mode="packed" heights="160" style="text-align:center; margin: 0px;">

File:HGBerry.jpg|Maj. Gen.<br />Hiram G. Berry, killed

File:Major General Charles Devens of 3rd Massachusetts Rifles Battalion and 15th Massachusetts Infantry Regiment in uniform - J.W. Black, 173 Washington St., Boston LCCN2016649627.jpg|Brig. Gen.<br />Charles Devens, wounded

File:Amiel Weeks Whipple.jpg|Brig. Gen.<br />Amiel W. Whipple, mortally wounded

File:Gershom Mott - Brady-Handy.jpg|Brig. Gen.<br />Gershom Mott, wounded

File:WilliamHays.jpg|Brig. Gen.<br />William Hays, wounded

File:Stonewall Jackson.jpg|Lt. Gen.<br />Stonewall Jackson, mortally wounded

File:Image of Lieutenant General A.P. Hill.jpg|Maj. Gen.<br />Ambrose P. Hill, wounded

File:Henry Heth.jpg|Brig. Gen.<br />Henry Heth, wounded

File:William Dorsey Pender.jpg|Brig. Gen.<br />William D. Pender, wounded

File:Samuel McGowan (general).jpg|Brig. Gen.<br />Samuel McGowan, wounded

File:Dodson Ramseur.jpg|Brig. Gen.<br />Stephen D. Ramseur, wounded

File:RFHokecommons.jpg|Brig. Gen.<br />Robert F. Hoke, wounded

File:Francis Redding Tillou Nichols.jpg|Brig. Gen.<br />Francis T. Nicholls, wounded

File:EAO'Neal.jpg|Brig. Gen.<br />Edward A. O'Neal, wounded

File:Paxton-Elisha Franklin.jpg|Brig. Gen.<br />Elisha F. Paxton, killed

File:Stapleton Crutchfield.JPG|Col.<br />Stapleton Crutchfield, wounded

</gallery>

|}

Lee, despite being outnumbered by a ratio of over two to one, won arguably his greatest victory of the war, sometimes described as his "perfect battle." But he paid a terrible price for it, taking more casualties than he had lost in any previous battle, including the stalemate at the Battle of Antietam. With only 60,000 men engaged, he suffered 13,303 casualties (1,665 killed, 9,081 wounded, 2,018 missing),

Of the 133,000 Union men engaged, 17,197 were casualties (1,606 killed, 9,672 wounded, 5,919 missing),

Assessment of Hooker

Hooker, who began the campaign believing he had "80 chances in 100 to be successful", lost the battle through miscommunication, the incompetence of some of his leading generals (most notably Howard and Stoneman, but also Sedgwick), but mostly through the collapse of his own confidence. Hooker's errors included abandoning his offensive push on May 1 and ordering Sickles to give up Hazel Grove and pull back on May 2. He also erred in his disposition of forces; despite Abraham Lincoln's exhortation, "this time put in all your men," some 40,000 men of the Army of the Potomac scarcely fired a shot. When later asked why he had ordered a halt to his advance on May 1, Hooker is reputed to have responded, "For the first time, I lost faith in Hooker." However, Stephen W. Sears has categorized this as a myth:

Sears's research has shown that Bigelow was quoting from a letter written in 1903 by an E. P. Halstead, who was on the staff of Doubleday's I Corps division. Sears concludes: